-- Logs begin at Wed 2022-08-24 06:58:56 UTC, end at Wed 2022-08-24 08:03:53 UTC. -- Aug 24 07:26:13.706968 dv-cinder-854273-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 07:26:13.715348 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 07:26:13 2022] *** Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: nodename: dv-cinder-854273-1-iscsi Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: machine: x86_64 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: clock source: unix Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: pcre jit disabled Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: detected number of CPU cores: 4 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: current working directory: / Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: detected binary path: /usr/bin/uwsgi-core Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: your processes number limit is 47680 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: your memory page size is 4096 bytes Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: detected max file descriptor number: 2048 Aug 24 07:26:13.717396 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: lock engine: pthread robust mutexes Aug 24 07:26:13.719151 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: thunder lock: enabled Aug 24 07:26:13.719151 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 07:26:13.719151 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 07:26:13.756946 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: Python main interpreter initialized at 0x5639ec5859c0 Aug 24 07:26:13.756946 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: python threads support enabled Aug 24 07:26:13.756946 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: your server socket listen backlog is limited to 100 connections Aug 24 07:26:13.756946 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: your mercy for graceful operations on workers is 90 seconds Aug 24 07:26:13.757266 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 07:26:13.757345 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: *** Operational MODE: preforking *** Aug 24 07:26:13.757618 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: *** uWSGI is running in multiple interpreter mode *** Aug 24 07:26:13.757618 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: spawned uWSGI master process (pid: 81670) Aug 24 07:26:13.757807 dv-cinder-854273-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 07:26:13.758341 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: spawned uWSGI worker 1 (pid: 81677, cores: 1) Aug 24 07:26:13.758666 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: spawned uWSGI worker 2 (pid: 81678, cores: 1) Aug 24 07:26:13.758827 dv-cinder-854273-1-iscsi devstack@keystone.service[81670]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 07:26:18.038945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:26:18.038945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: from cryptography.utils import int_from_bytes Aug 24 07:26:18.039756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:26:18.039756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: from cryptography.utils import int_from_bytes Aug 24 07:26:18.039903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:26:18.039903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: from cryptography.utils import int_from_bytes Aug 24 07:26:18.039903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:26:18.039903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: from cryptography.utils import int_from_bytes Aug 24 07:26:18.090199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:26:18.090365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:26:18.090455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:26:18.090540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:26:18.090620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] command line args: [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:26:18.090753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] command line args: [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:26:18.090897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:26:18.091016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ================================================================================ {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:26:18.091104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:26:18.091184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ================================================================================ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:26:18.091260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] admin_token = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] admin_token = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] config_dir = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_dir = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] config_source = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.091937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_source = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] debug = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] debug = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.092930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_config_append = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_config_append = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.093984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_dir = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_dir = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_file = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.094913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.095971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.096894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.097816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.098066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.098247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] publish_errors = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.098332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.098478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] publish_errors = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.098558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.099978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] standard_threads = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] standard_threads = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.100948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] use_journal = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] use_json = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_journal = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] use_stderr = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_json = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] use_syslog = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_stderr = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_syslog = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:26:18.101949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.102962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.103914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.104871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.105971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.106973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107810 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.107965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.108754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.109964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.110035 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.110799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.110884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.110956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.111981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.112991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.113897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.114011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.114132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.114250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.114964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.115990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.116978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.117925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.118920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.119930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.120876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.121975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.122975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.123924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.124937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.125999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.126880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.external = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.external = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.127877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.password = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.password = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] auth.token = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.token = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.128955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.129925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.130920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.131946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.132926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.133944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.134916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.135857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.136974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.137934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.138972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.139918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.140897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.141882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.142972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.143998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.144886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.145969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.146903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.147932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.148955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.149970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.150883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.151993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.152874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.153873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.154952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.155992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.156944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.157974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.158969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.159913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] role.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] role.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.160845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.161013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.161632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.161792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.161871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.162880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.163837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.164981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.165884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.166891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.167782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.168894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.169769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.170994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.171868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.172970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.173970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.connection = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.174919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.175952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.176708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.177404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.177495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.177613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.177736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.177875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.178880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81678) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:26:18.179628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:26:18.179628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:26:18.208025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.208850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.209007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.209147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.209280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.209412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.217516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.217710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.245343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.245579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.245708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.245901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.299799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.300429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.349573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.349812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.353310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.353549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.506814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.507802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.508014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.508152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.508291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.531322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.531559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.531690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.531822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.531966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.532890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.533018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.533018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.544226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.544630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.556980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.557987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.558069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.558200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81678) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.558280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.558414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:26:18.575153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.575362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:26:18.575707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.575788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.575909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:26:18.576010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:26:18.576186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.576268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.576451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:26:18.576530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.576727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.576963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.577045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.577230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.577488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.577756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.587235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.587522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.587773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.587859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.588057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.588154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.588331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.588620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.595058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.595704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.597951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.598238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.598523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.598604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.599377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.599541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.599666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.599924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.600859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.601206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.601282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.601398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.601480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.601759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.602270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.602554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.602851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.623949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.625414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.625613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.625695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.627106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.627393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.630454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.630761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.631052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.632184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.632470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.632757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.637371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.637660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.639178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.639467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.641774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.642228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.642664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.642972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.643253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.643531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.643762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.643845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.644063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.644142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.644515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.644889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.645276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.646182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.646566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.646976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.662324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.662606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.666521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.669512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.669731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.669950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.670034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.673850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.674264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.677453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.678793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.679219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.680714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.681018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.681316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.681614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.681898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.682191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.683602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.686770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.689988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.690297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.690594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.690920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.691471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.691761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.698309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.698609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.698973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.699245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.706143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.707497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.707699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.707782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.708498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.708585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.708824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.711181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.711468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.711764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.712073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.712360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.716016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.717551 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.717827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.720499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.720791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.721086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.721390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.721672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.724506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.724796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.725085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.725373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.725660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.725944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.726236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.733642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.733923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.734206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.734487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.734861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.735145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.735426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.742192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.745255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.748167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.749611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.751111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.751336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.751419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.751682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.754357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.757866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.758678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.759337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.760152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.760814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.761098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.761396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.763066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.763362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.763651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.763958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.767900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.769342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.772020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.772279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.772380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.772570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.772652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.772856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.773141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.781078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.781355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.781637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.781742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81678) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.782030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.782317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.782598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.782910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.783199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.783478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.783749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.784038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.784311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.784596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.784863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.785143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.785407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81678) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.788568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:26:18.788852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.789133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.789410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.789707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.789982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.790257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.790527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.790852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.791322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.791405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.791658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.791934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.792197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:26:18.820518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639ec5859c0 pid: 81678 (default app) Aug 24 07:26:18.823896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeeac8-0101-4eb9-a3bf-fddbbaea220a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:18.824185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeeac8-0101-4eb9-a3bf-fddbbaea220a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:18.824461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeeac8-0101-4eb9-a3bf-fddbbaea220a None None] PATH_INFO: `/v3` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:18.827236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639ec5859c0 pid: 81677 (default app) Aug 24 07:26:18.829669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Aug 24 07:26:18 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:25.406631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:25.407164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:25.407247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:25.411499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:26:25.417980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:25.418345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:25.418524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:25.489790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: INFO dbcounter [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] Registered counter for database keystone Aug 24 07:26:25.509146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG oslo_db.sqlalchemy.engines [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81678) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:26:25.515489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writer thread running {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:26:25.695756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG passlib.handlers.bcrypt [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81678) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:26:25.696091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG passlib.handlers.bcrypt [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81678) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:26:25.749835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:25.751461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:25.881386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 07:26:26.325169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:26.393952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: INFO dbcounter [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] Registered counter for database keystone Aug 24 07:26:26.415836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG oslo_db.sqlalchemy.engines [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81677) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:26:26.422284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writer thread running {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:26:26.543044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265ed8-8aff-4977-9b81-b26125e3fbb2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:26.547323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:26.547937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:26.548213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:26.548481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:26.556457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:26.608239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:26:26.608572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379c770-33b1-4dfe-a640-0d5c578967df admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:26.618321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Aug 24 07:26:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:26:26.624322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:26.639268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100feab-8a84-41c6-a92f-839358a12880 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:26.643611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:26.645307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:26.645590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:26.645863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] PATH_INFO: `/v3/projects/86fab351db604d94a6744cf147045d7e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:26.647448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] RBAC: Authorizing `identity:get_project(project_id=86fab351db604d94a6744cf147045d7e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:26.699952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:26:26.700291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a0a4-ae94-494c-b7e0-0d22baf70de9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:26.704751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Aug 24 07:26:26 2022] GET /identity/v3/projects/86fab351db604d94a6744cf147045d7e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:28.577894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb593fd-b7ea-4fc9-ac88-7dc3f4fda938 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:28.578480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb593fd-b7ea-4fc9-ac88-7dc3f4fda938 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:28.580974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb593fd-b7ea-4fc9-ac88-7dc3f4fda938 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:28.585134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:26:28.590659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:28.591077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:28.591254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:28.611305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:28.612827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:28.621296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 07:26:28.885697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb593fd-b7ea-4fc9-ac88-7dc3f4fda938 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:28.898137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb593fd-b7ea-4fc9-ac88-7dc3f4fda938 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:28.902223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:28.902845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:28.903128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:28.903401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:28.912054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:28.913818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baba466-e990-4b74-94e1-ce95ad94ff8f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:28.922301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Aug 24 07:26:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:26:28.927574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:28.936591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd757-b38f-43ed-b859-09d882ba3b27 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:28.941145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:28.942277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:28.942556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:28.942871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:28.944173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:28.945980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4568f-cd4e-464c-9971-7e30fc1e283d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:28.947522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Aug 24 07:26:28 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:30.917485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea261-f183-4a16-bb0b-8bd01ea65df2 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:30.918022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea261-f183-4a16-bb0b-8bd01ea65df2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:30.918022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea261-f183-4a16-bb0b-8bd01ea65df2 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:30.919366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:30.925454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0af8c2-d67d-47dc-ad1b-962022daf30d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:30.925742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0af8c2-d67d-47dc-ad1b-962022daf30d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:30.925836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:30.926199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0af8c2-d67d-47dc-ad1b-962022daf30d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:30.926275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:30.926275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:30.927276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:30.932919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:30.933357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:30.933620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:30.945087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:30.946259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:30.975322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG passlib.handlers.bcrypt [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81677) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:26:30.975735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG passlib.handlers.bcrypt [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81677) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:26:31.021056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:31.022633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:31.023824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:31.119666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:31.293013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.316556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.384933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.398321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5dfa4-0dd0-48cd-a667-16fef47a15d5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.431730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.432362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.432647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.432956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.445409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:31.453822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:31.454786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:31.455673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:31.461310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.488240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e51e5-82e1-4f0d-a206-ba00f5047bfa None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.490602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.527181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.528059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.528339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.528615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.541292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.559743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307ca9f-0e28-49a9-92e7-6dcf0e8b05c9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.593343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.593955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.594242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.594513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.595208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:31.595510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:31.596897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:31.599428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:31.600995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:31.609195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:31.615283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.699112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:26:31.704677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.707936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.714540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17071-06c1-4342-bb22-347b65dd8508 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.748001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.748621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.748901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.749173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.750299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:31.751775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:31.753329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:31.758806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.772412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.803185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d59c9-9a9a-4794-be3c-eb79a925f1f2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.836428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.837211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.837496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.837772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.856373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:31.857635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaee46-5172-4241-8de5-57ce96b0ef01 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:31.857968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:26:31.876306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:31.877279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:26:31.878075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:26:31.878075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:26:31.878075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:31.878075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:26:31.883657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.892544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:31.893174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:31.893454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:31.893732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:31.895427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:31.896876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:31.898555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:31.904078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:31.942163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.983017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:31.985041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be057-e791-43f4-9742-a239153fbb70 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:32.003080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76164-b70e-4336-88ad-96db8a738689 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:32.020500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:32.021112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:32.021397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:32.021672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:32.022324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:32.022628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:32.024063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fa2d4-a4b3-4c20-b3c0-ee3bf089ae33 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:32.037819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:32.038427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:32.038733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:32.039000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:32.039728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.domain.id=default, user.id=11b0f239f5044e6880117c0daf3fe327) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:32.040032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:32.041436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:32.041535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701454e-1e58-436c-b562-a1e09f550480 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:32.052552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Aug 24 07:26:31 2022] GET /identity/v3/role_assignments?user.id=11b0f239f5044e6880117c0daf3fe327&scope.domain.id=default&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:34.229511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d4bc-78df-496a-ba9e-8e0895ac93af None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.230468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d4bc-78df-496a-ba9e-8e0895ac93af None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.230994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d4bc-78df-496a-ba9e-8e0895ac93af None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.232905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:34.239246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1f19-2063-4e81-9fc4-e452dd0fddb3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.239674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1f19-2063-4e81-9fc4-e452dd0fddb3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.239848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1f19-2063-4e81-9fc4-e452dd0fddb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.257899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27f544-14d5-4018-b60a-894bc168996e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.258210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27f544-14d5-4018-b60a-894bc168996e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.258466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27f544-14d5-4018-b60a-894bc168996e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.260536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 12/23] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:26:34.266793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-15ef1f19-2063-4e81-9fc4-e452dd0fddb3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:34.267999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-15ef1f19-2063-4e81-9fc4-e452dd0fddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:34.273873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41fff-4d49-445e-9362-0ca40eac7a0c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.274163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41fff-4d49-445e-9362-0ca40eac7a0c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.274449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41fff-4d49-445e-9362-0ca40eac7a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.302347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e2d41fff-4d49-445e-9362-0ca40eac7a0c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:34.303614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e2d41fff-4d49-445e-9362-0ca40eac7a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:34.401680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 12/24] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:34.448168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:34.610110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef88-db81-4a8e-9a1e-87c436ab0663 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.610387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef88-db81-4a8e-9a1e-87c436ab0663 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.610917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef88-db81-4a8e-9a1e-87c436ab0663 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.612473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 13/26] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:34.617843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:34.618136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:34.618420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:34.648388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:34.649588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:34.716677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:34.748855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:34.815275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fef88-db81-4a8e-9a1e-87c436ab0663 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:34.847868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:34.931129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:34.957627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fef88-db81-4a8e-9a1e-87c436ab0663 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.031609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.032251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.032530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.032805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.053782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.055289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.077809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4cf17-3602-4647-b29e-e800b2858ab6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ee2bb48c-a1ed-491b-aece-72d9a979a7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:26:35.078454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:35.079323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:26:35.080543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:26:35.080543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:26:35.080543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:35.080543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 14/28] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:35.083135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44dfa-f62b-4135-ab72-acf9e3acaf1d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.083135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44dfa-f62b-4135-ab72-acf9e3acaf1d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.083135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44dfa-f62b-4135-ab72-acf9e3acaf1d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.083135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 15/29] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:35.096235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.096522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.096802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.119897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.120455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.120733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.121001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.125350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:35.126572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:35.133807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:35.140465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:35.142591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 15/30] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:26:35.144777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.180940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:35.206046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 16/31] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:35.206943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.251908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb369185-de7e-48f2-b6f7-2bf527383274 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.284346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:35.291615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.292521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.293010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.293474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.294386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:35.294914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.296534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.310009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 16/32] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:35.316680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.336381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6088e5-16e1-4281-828d-0810af3b5211 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.374358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.375001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.375288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.375562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.376238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.377697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.425048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:35.431832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e99535-c319-4a79-be7c-4c92b4c06acf None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.480451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.481117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.481403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.481831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.482507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:35.482839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.484284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.503835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:35.510922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.525140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 17/34] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 24 07:26:35 2022] POST /identity/v3/roles => generated 219 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:35.526399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.655650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438932f8-dfde-4e2e-90cf-0c3ba170420c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.722765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028752-68ad-4e01-9650-09e78ba4e872 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.725147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.726656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.726656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.726656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.727600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.729201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.743446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:35.767427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.768176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.768464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.768741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.770645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.772232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e4d5f-d67f-4bb8-b651-a204cd0cdb20 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.773836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:35 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:35.775077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266512-88c1-41ef-a86c-8260c155f4d3 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.775372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266512-88c1-41ef-a86c-8260c155f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.775637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266512-88c1-41ef-a86c-8260c155f4d3 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.776912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 19/36] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:35.777609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266512-88c1-41ef-a86c-8260c155f4d3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.890723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 18/37] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:26:35 2022] POST /identity/v3/projects => generated 301 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:35.890723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:35.898671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:35.900255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266512-88c1-41ef-a86c-8260c155f4d3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:35.949491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:35.950237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.950924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.951299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:35.952070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:35.954952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52157-e545-4dad-bb6b-440e6b5f5d47 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:35.977016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 20/38] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 24 07:26:35 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:35.978505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c05fa-fe90-4884-8cde-c71f2b03c834 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:35.978853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c05fa-fe90-4884-8cde-c71f2b03c834 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:35.979185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c05fa-fe90-4884-8cde-c71f2b03c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.014422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a16c05fa-fe90-4884-8cde-c71f2b03c834 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:36.020650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a16c05fa-fe90-4884-8cde-c71f2b03c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:36.048831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9a87-c26f-4855-947e-b1ec70d1b24d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:36.099548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:36.109422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:36.110451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.110973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.111457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.112983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:36.114812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:36.116688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 19/39] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:36.122912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:36.142146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 21/40] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:36.217562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:36.243151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606a9d3-0066-410d-88d7-31a0d0fdf3e6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:36.278284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:36.279072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.279260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.279538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] PATH_INFO: `/v3/domains/default/users/11b0f239f5044e6880117c0daf3fe327/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.281848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=11b0f239f5044e6880117c0daf3fe327, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:36.283281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebcaeb-a971-4626-a489-ce4084d0bec3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:36.301996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Aug 24 07:26:36 2022] PUT /identity/v3/domains/default/users/11b0f239f5044e6880117c0daf3fe327/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:36.359694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e88-bddc-4236-b432-5f2b0f3cd661 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.360282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e88-bddc-4236-b432-5f2b0f3cd661 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.360470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e88-bddc-4236-b432-5f2b0f3cd661 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.362001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 22/42] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:36.368666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.368955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.369247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.393501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:36.397009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:36.432275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e88-bddc-4236-b432-5f2b0f3cd661 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:36.473819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:36.549277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e88-bddc-4236-b432-5f2b0f3cd661 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:36.586952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:36.587909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.588907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.589186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.589868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:36.591295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:36.606848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 24 07:26:36 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:26:36.742763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:36.812930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:36.850340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fea4c-9961-4754-9a6b-25515c367315 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:36.884096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:36.884723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:36.885005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:36.885280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:36.886379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:36.887858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729e601-65b1-4348-8254-eb5b968c8055 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:36.889406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:36.894958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:36.930066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:36.999359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ea0c-6098-4200-b08d-97ac9fcef67f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:37.031638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:37.032246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:37.032530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:37.032805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:37.033464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:37.035474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77d4b6-1824-4711-8aa7-2d2dee779f4e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:37.063186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:26:36 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:38.211639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb6ac-67bb-4cd0-ab7b-e837f1333a67 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:38.212156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb6ac-67bb-4cd0-ab7b-e837f1333a67 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:38.212258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb6ac-67bb-4cd0-ab7b-e837f1333a67 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:38.213529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:38.218926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:38.219335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:38.219578 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:38.238252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:38.239480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:38.411227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:38.679884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb6ac-67bb-4cd0-ab7b-e837f1333a67 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:38.884997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb6ac-67bb-4cd0-ab7b-e837f1333a67 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:38.918456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:38.919119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:38.919402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:38.919676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:38.922260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:38.930295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f88c3dd0-d10c-44c8-bd7f-9e6b01c9b3ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:38.935125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:38.936112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:38.937219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:38.942269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:38.960842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:38.967283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b990c-f2b3-4059-b324-2136d634e43c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:38.967572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b990c-f2b3-4059-b324-2136d634e43c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:38.967850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b990c-f2b3-4059-b324-2136d634e43c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:38.969201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 25/50] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:26:38.974669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:38.974996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:38.975276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:38.993219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:38.994441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:39.047133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513a3b5-0b9f-4eb7-92a2-a498ea26fea8 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.060964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 26/51] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:39.081929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.082548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.082875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.083158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.083862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:39.084169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.085562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.093540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 26/52] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:39.099832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:39.197717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1602ed-ad00-410b-9572-2adab6db6c34 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.222686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:39.231837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.232472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.232757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.233036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.234171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.235655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.237221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:39 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:39.243481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:39.324861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:39.325884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:39.348434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ee3ea-bf58-42d2-90ca-f0d0a7cfa5bb None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.381316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.381949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.382241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.382522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.383659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.385094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.386713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:39.392427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:39.429960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2294700-2785-45cf-a33f-1e73b4e05084 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.465352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.465907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.466184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.466456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.467678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.469151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.470953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:39.476223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:39.492010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de0231-4f61-4cd2-8c86-77753e89a7ba None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.526253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.526976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.527325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.527668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.528403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.domain.id=default, user.id=11b0f239f5044e6880117c0daf3fe327) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:39.528711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.530142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eaf454-b7ae-4bae-a829-eb49c47d8b0f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.536240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 28/56] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Aug 24 07:26:39 2022] GET /identity/v3/role_assignments?user.id=11b0f239f5044e6880117c0daf3fe327&scope.domain.id=default&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:39.536811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:39.579932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41bd67-57f9-4631-ae9f-2007a11f93f7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:39.620926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:39.621556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:39.621843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:39.622121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:39.622816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:39.624296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc0dfa-cf7e-4938-ac6d-571a10b8e075 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:39.653388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 29/57] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Aug 24 07:26:39 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:26:41.549212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df2295-7aa9-42b0-88b9-eac18ea8d417 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:41.549768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df2295-7aa9-42b0-88b9-eac18ea8d417 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:41.549768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df2295-7aa9-42b0-88b9-eac18ea8d417 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:41.551690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:41.556981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:41.557391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:41.557533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:41.575415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:41.576601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:41.641567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:41.919682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df2295-7aa9-42b0-88b9-eac18ea8d417 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:41.932435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:42.021983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df2295-7aa9-42b0-88b9-eac18ea8d417 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:42.054641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:42.055297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:42.055607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:42.055882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:42.056995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:42.058442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69b58e-c0d0-4e0b-99c9-aed8d55045fb None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:42.060118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:42.066435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:42.109900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:42.162877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e526-bde0-4e20-b90a-2aca5c5685b3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:42.195698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:42.196247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:42.196528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:42.196804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:42.197705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:42.199200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b76b3c-6800-4556-8a83-f9cbc17b9c8e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:42.224912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:26:42 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:45.167778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cceb2-8cc7-44bc-9109-4366f3c53496 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.168825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cceb2-8cc7-44bc-9109-4366f3c53496 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.169326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cceb2-8cc7-44bc-9109-4366f3c53496 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.171134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:45.184230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b7be0-f445-4363-b645-6637d654ad33 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.185070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b7be0-f445-4363-b645-6637d654ad33 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.185550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b7be0-f445-4363-b645-6637d654ad33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.215674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-128b7be0-f445-4363-b645-6637d654ad33 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:45.217187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-128b7be0-f445-4363-b645-6637d654ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:45.300526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 32/63] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:45.332086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e8dd-5742-4984-82e5-ac72fe23932f None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.332678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e8dd-5742-4984-82e5-ac72fe23932f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.333206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e8dd-5742-4984-82e5-ac72fe23932f None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.334992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 32/64] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:45.340395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd58d-c4fe-472b-aee8-7a3d3a57f2dc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.340893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd58d-c4fe-472b-aee8-7a3d3a57f2dc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.341362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd58d-c4fe-472b-aee8-7a3d3a57f2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.370952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-827fd58d-c4fe-472b-aee8-7a3d3a57f2dc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:45.372469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-827fd58d-c4fe-472b-aee8-7a3d3a57f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:45.405554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:45.469433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 33/65] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:45.506898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef8989-d1f2-4128-86f4-9b2ae101504a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.507521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef8989-d1f2-4128-86f4-9b2ae101504a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.508018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef8989-d1f2-4128-86f4-9b2ae101504a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.509801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 33/66] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:45.514766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7111-4849-46a5-a96b-96700c2d9bde None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.515301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7111-4849-46a5-a96b-96700c2d9bde None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.515772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7111-4849-46a5-a96b-96700c2d9bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.539646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-282a7111-4849-46a5-a96b-96700c2d9bde None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:45.541124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-282a7111-4849-46a5-a96b-96700c2d9bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:45.634335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:45.651455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdac3-a56e-4bba-9728-6c5c491ecf3b None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.652076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdac3-a56e-4bba-9728-6c5c491ecf3b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.652544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdac3-a56e-4bba-9728-6c5c491ecf3b None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.654064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 34/68] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:45.660209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:45.660499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:45.660778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:45.685485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:45.686784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:45.775728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdac3-a56e-4bba-9728-6c5c491ecf3b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:45.789864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 35/69] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:45.863727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:45.890270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:45.898531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:45.948349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdac3-a56e-4bba-9728-6c5c491ecf3b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:45.993594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9ae1f-bf4a-413e-9c40-574453d3e793 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.004318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.005259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.006116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.006539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.022297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:46.028977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:46.030022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:46.030965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:46.032900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:26:46.033809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.039952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.040771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.041208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.041620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.055423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:46.062174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:46.063209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:46.064023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:46.067820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 36/71] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:46.073318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.146802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197bb9-975d-41c6-bd59-8f1a5e04a0f4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.160723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.179349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b650026-93e4-4273-b432-e919b643609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.179978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b650026-93e4-4273-b432-e919b643609b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.180103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700add1f-bf95-45fa-b0ca-3b5996d9d324 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.180256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b650026-93e4-4273-b432-e919b643609b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.180528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b650026-93e4-4273-b432-e919b643609b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.187201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.193023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b650026-93e4-4273-b432-e919b643609b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-7b650026-93e4-4273-b432-e919b643609b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:46.197740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:46.198625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:46.199437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:46.200065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b650026-93e4-4273-b432-e919b643609b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.218581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.219216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.219502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.219779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.220445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:46.220751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.222162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.230205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.234996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.298569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b650026-93e4-4273-b432-e919b643609b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.329318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f2ee-6ab3-4a1c-90dc-1a88c213466e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.332003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.332611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.332894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.333169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.333828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:46.334131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.335546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f686-1051-416d-a755-e63e519d2601 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.343384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.348071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82213e1-d1e8-4011-af63-8be73b5ca923 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.348071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82213e1-d1e8-4011-af63-8be73b5ca923 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.348071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82213e1-d1e8-4011-af63-8be73b5ca923 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.348071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 38/75] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:46.348071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82213e1-d1e8-4011-af63-8be73b5ca923 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.367371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.368007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.368294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.368581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.379194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:46.383930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:46.384793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:46.391295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.392123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.406902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.448917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82213e1-d1e8-4011-af63-8be73b5ca923 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.483941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.485032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.485551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.486063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.487886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:46.488440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.489620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd121f-f2b8-40ee-928b-f1b1f4b33589 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.491101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.500306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.500933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.525796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.526433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.526743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.527027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.541680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.543177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14257150-da0a-4848-830b-13a016fb9eda None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.544787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.545928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.546219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.546502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.570073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:46.571403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:46.604496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520bef-1b9c-481f-9ec0-77f7f53a5352 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.626915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.628090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.639304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 40/79] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:46.654815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.655920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.656629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.657064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.657346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.659354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.660813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.663286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.668167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.743518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6ad1-f710-4432-867c-78cebb52e8dd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.771823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76865925-7665-4c6d-b4b2-2a180ac03485 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.780245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.780863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.781148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.781432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.782213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:46.782523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.784124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.792565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.798463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.806680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.807330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.807613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.807888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.809069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:46.810518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:46.812112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 41/82] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:46.812863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:46.856752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.882180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:46.914567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec885-6c3c-43b3-9164-5a542ba14eba None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.928086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ad609-4b4d-4155-8ed8-ddebdb3485ff None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:46.948075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.948688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.948969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.949245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:46.962510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:46.963223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:46.963579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:46.963929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.015511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.016987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.031665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.033154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.067201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.091331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.115312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 42/83] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:47.116080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.135285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 42/84] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:47.136129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.219238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358aaf-61fc-4d92-a959-321f818ab0bc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.233276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3d6e3-46c2-45f8-976c-c4c2c73f1170 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.253264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.253887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.254171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.254437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.255585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.257020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.258572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 43/85] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:47.263411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.267648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.268814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.269097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.269370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.280355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.296713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.338839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.340447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.357894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943734b-45c4-4c75-9b3d-2eea18e75364 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.390224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.390805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.391084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.391355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.392447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.393945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.395614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 44/86] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:47.396434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.443702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 43/87] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:47.445818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.482930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.495835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954beef-2cd7-4860-8700-f3ded905748a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.503195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.528769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.529398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.529679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.529955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.531152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.532663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.534846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 45/88] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:47.535640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.544631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8e05-54e9-4eb0-b071-b0cba6201cc3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.581534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.582186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.582475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.582787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.583950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.585417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.587099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 44/89] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:47.587854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.639828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afdb1-67c5-4cad-a48a-79567c864908 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.673836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.674398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.674682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.674988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.686581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aeb6-1376-4b29-97e8-5076b24fd917 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.701385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.713140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.721994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.722607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.722923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.723206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.723856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.725302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.744172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.745767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.755096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 45/90] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:26:47 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:47.756457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.855724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 46/91] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:47.861330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.877035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a12c-c69c-4419-98ce-0ffcd2f233f3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.922855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:47.923501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:47.923788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:47.924125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:47.925819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:47.927309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:47.928781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:47.929314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:47.930738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:47.979592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e2ca-726c-4b45-9b66-65f9371fa39f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:47.996782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:48.026191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:48.026853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:48.027338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:48.027685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:48.028432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:48.028749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:48.030183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:48.038749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:48.039555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:48.071637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbfa95-7d1f-4b6b-a3fb-54a8bc15d703 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:48.122281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:48.123326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:48.123807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:48.124378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:48.125437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=11b0f239f5044e6880117c0daf3fe327) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:48.125973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:48.128369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:48.135209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 47/94] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:47 2022] GET /identity/v3/role_assignments?user.id=11b0f239f5044e6880117c0daf3fe327&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:48.136398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:48.173811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407996-b9ac-4214-a049-fa0eba4fa438 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:48.204316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:48.222062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:48.223228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:48.223523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:48.223803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] PATH_INFO: `/v3/projects/0484384e3bc142bebbd15edb6edf51a2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:48.224995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] RBAC: Authorizing `identity:get_project(project_id=0484384e3bc142bebbd15edb6edf51a2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:48.226639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:48.228407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 48/95] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:48 2022] GET /identity/v3/projects/0484384e3bc142bebbd15edb6edf51a2 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:48.234632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:48.340687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4052-81e4-49ac-9bc8-661875c3390c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:48.360984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefaebd-a89b-4e87-946f-4e711c52384d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:48.371029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:48.401296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:48.401933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:48.402214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:48.402487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:48.403451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=0484384e3bc142bebbd15edb6edf51a2, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:48.403938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:48.405616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07c65-e766-44db-9a01-8d815d458b92 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:48.419212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 49/96] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:48 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=0484384e3bc142bebbd15edb6edf51a2&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:48.421158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:48.421865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:48.422150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:48.422425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:48.423226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:48.423536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:48.425724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8f41-7b10-4430-b237-a16fb62c9165 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:48.439942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 48/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:48 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=428b94d9bca848c1a1180b457c176272 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:50.481258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e0da0-7301-4d96-93e9-319282b8087f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.481767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e0da0-7301-4d96-93e9-319282b8087f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.481838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e0da0-7301-4d96-93e9-319282b8087f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.483321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 50/98] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:50.500753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998742-1912-4dd3-bb0f-b433282c9879 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.501707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998742-1912-4dd3-bb0f-b433282c9879 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.502184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998742-1912-4dd3-bb0f-b433282c9879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.530419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2b998742-1912-4dd3-bb0f-b433282c9879 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:50.531993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2b998742-1912-4dd3-bb0f-b433282c9879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:50.630922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:50.825274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d66f-fe9f-41a7-8a00-ec352560f5f8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.825613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d66f-fe9f-41a7-8a00-ec352560f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.825779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d66f-fe9f-41a7-8a00-ec352560f5f8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.827389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 51/100] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:50.832881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9410-2212-41e8-890a-1b2e0c52c057 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.833468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9410-2212-41e8-890a-1b2e0c52c057 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.833943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9410-2212-41e8-890a-1b2e0c52c057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.857408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4636c8-bf01-490a-b7f3-1e4aeda43d10 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.857706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4636c8-bf01-490a-b7f3-1e4aeda43d10 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.857976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4636c8-bf01-490a-b7f3-1e4aeda43d10 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.859384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 52/101] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:26:50.859761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9d7a9410-2212-41e8-890a-1b2e0c52c057 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:50.861115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9d7a9410-2212-41e8-890a-1b2e0c52c057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:50.865174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875513e7-35aa-4f0e-b415-5a46dfd7092a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:50.865570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875513e7-35aa-4f0e-b415-5a46dfd7092a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:50.865974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875513e7-35aa-4f0e-b415-5a46dfd7092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:50.896318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-875513e7-35aa-4f0e-b415-5a46dfd7092a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:50.898676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-875513e7-35aa-4f0e-b415-5a46dfd7092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:50.984504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 50/102] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:51.004024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.014289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 53/103] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:51.094023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3e20-ede5-4882-83c7-ef8b2c7fa91d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.094315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3e20-ede5-4882-83c7-ef8b2c7fa91d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.094496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3e20-ede5-4882-83c7-ef8b2c7fa91d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.096118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 51/104] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:51.106152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de088-6837-4285-ad89-219a748b15a0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.106685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de088-6837-4285-ad89-219a748b15a0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.107187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de088-6837-4285-ad89-219a748b15a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.131638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0c5de088-6837-4285-ad89-219a748b15a0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:51.133147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0c5de088-6837-4285-ad89-219a748b15a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:51.167903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3e20-ede5-4882-83c7-ef8b2c7fa91d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.239571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.241778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 54/105] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:51.243860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d661a-0184-4098-83d6-7009e76048f2 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.244292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d661a-0184-4098-83d6-7009e76048f2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.245077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d661a-0184-4098-83d6-7009e76048f2 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.250902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 55/106] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:51.253629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.253915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.254193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.273733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:51.275010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:51.313566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3e20-ede5-4882-83c7-ef8b2c7fa91d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.354724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 56/107] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:51.354895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.354895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.354895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.354895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.372626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:51.377512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:51.378338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:51.379113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 52/108] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:26:51.386186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.441026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.518945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b8be5-3b73-458c-a45d-164cbd238821 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.554591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.555245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.555528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.555856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.556533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:51.556888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:51.557934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.560763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:51.570347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 57/109] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:51.571184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.678390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aa2ba-bafe-4ede-96d8-1de36e77ebbc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.679063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.681758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.683911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71857b-c860-4ee1-84bb-0b03b9bcfd08 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.715193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.715831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.716117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.716389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.717667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:51.719145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:51.720940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 58/110] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:51.721515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.725754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.726444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.726762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.727048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.738851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:51.743647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:51.744456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:51.745201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 53/111] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:26:51.746846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.831687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4812f-4ea6-44c5-9fde-90d8f726e15e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.852160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402f43-6a52-4937-8938-c91e37d2e117 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:51.869345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.869969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.870253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.870527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] PATH_INFO: `/v3/users/29a8ffd181d848dcb0dc05853003dc6d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.883563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=29a8ffd181d848dcb0dc05853003dc6d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:51.885027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:51.886107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:51.890990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 59/112] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/users/29a8ffd181d848dcb0dc05853003dc6d => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:51.891146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:51.891146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:51.891146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:51.891348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.901445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:51.906254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:51.907141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 54/113] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:51.909444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:51.921115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.930913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:51.991214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3419ab-e822-4647-ba5d-b362320c5fe4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.010933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755258f-82fb-4eee-bb34-1ab383bd0d74 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.024925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.025555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.025833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.026103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.036280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:52.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:52.042045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:52.042860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 60/114] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:52.043579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.050949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.051564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.051842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.052114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.052756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.054488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2fb9d-1b82-4f62-8454-6e78e1a8b83e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.081429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 55/115] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:26:51 2022] POST /identity/v3/projects => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:52.082653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ca1fa-2700-4352-b4bb-511456240441 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.082974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ca1fa-2700-4352-b4bb-511456240441 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.083250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ca1fa-2700-4352-b4bb-511456240441 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.084630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 56/116] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:52.085169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ca1fa-2700-4352-b4bb-511456240441 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.163767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.176001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.177567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282bd8f-c75b-444f-b6c8-b639f442a0ea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.212973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ca1fa-2700-4352-b4bb-511456240441 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.219783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.220405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.220697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.221010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.221721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:52.222075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.223437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.234256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 61/117] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.235121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.255573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.256193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.256474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.256783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.333819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.335359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543d72-42e3-4386-bb5f-a39264316969 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.360730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cc65-e341-4ba7-88f5-ad8772dafc34 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.402812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.403314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.403665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.403949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.404622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:52.404927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.405921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.406316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.420186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 62/118] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.420186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.436117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.448048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 57/119] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:52.449317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.449606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.449880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.468428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:52.469549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:52.525840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa382557-cb62-4bf6-97b6-1f9b5668019d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.541152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 58/120] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:52.542252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.564408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.565686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.566070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.566450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.567403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:52.567870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.569448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.578863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 63/121] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.580099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.671975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.676168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa16424-32e8-4b5f-bbb3-c0c07316d2bb None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.681203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.713477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5bcbb-d874-4cd3-8d70-b85a0b2e16aa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.713779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.714532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.714955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.715333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.716612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.718197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.724539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 59/122] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.724539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.761681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.762515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.762937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.763487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.764577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.766202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.768536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 64/123] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.768870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.846787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f9ae-97bf-4576-933b-35b046205f37 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.880164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016188-9094-440c-89a3-4188e9a566a0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:52.882303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.882969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.883261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.883536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.884686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.886140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.888093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 60/124] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.888758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.918222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:52.918897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:52.919179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:52.919454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:52.920572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:52.922015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:52.923755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 65/125] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:52.924927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.929079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:52.945392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:52.996681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1ef8-3171-47b4-87db-e50e49aeb3ec None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.034990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.035641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.035924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.036207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.037327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.038793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.038981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79de4c-87f9-4a15-8ab5-a30248485f0f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.040434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 61/126] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:53.045967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.075037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.075681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.075967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.076251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.077398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.078902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.081085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 66/127] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:52 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:53.086442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.152573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f51c0-ba43-4f02-aef7-ea0dac763c5a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.155704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.176525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.191898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.192535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.192818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.193149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.199167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c22bc-0718-44d3-abc0-c27f728952ad None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.205963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:53.210939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:53.211794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 62/128] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:26:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:53.215393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.241204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.241817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.242099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.242373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.243169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=29a8ffd181d848dcb0dc05853003dc6d) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:53.243478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.244914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.258813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 67/129] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:53 2022] GET /identity/v3/role_assignments?user.id=29a8ffd181d848dcb0dc05853003dc6d&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:53.258813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.354878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9780cb-afa3-442b-bb56-42aed35c4e44 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.391069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01062f2-2040-459a-a469-b1cfb3ce4d86 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.396474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.397143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.397427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.397703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] PATH_INFO: `/v3/projects/0484384e3bc142bebbd15edb6edf51a2` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.399419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] RBAC: Authorizing `identity:get_project(project_id=0484384e3bc142bebbd15edb6edf51a2)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.400916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.406721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 63/130] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:53 2022] GET /identity/v3/projects/0484384e3bc142bebbd15edb6edf51a2 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:53.406721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.441715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.442915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.443829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.444115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.444396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/57a045168aa44e0386b65064bd0ea153/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.446534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, user_id=57a045168aa44e0386b65064bd0ea153, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.449134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.467823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 68/131] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:26:53 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/57a045168aa44e0386b65064bd0ea153/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:53.468748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.475411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.561865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5dd4-f31f-4895-a34c-978a645ffafe None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.600557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.601286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.601572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.601842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/11b0f239f5044e6880117c0daf3fe327/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.603872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, user_id=11b0f239f5044e6880117c0daf3fe327, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.605434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.630787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 64/132] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:26:53 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/11b0f239f5044e6880117c0daf3fe327/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:53.632840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.704391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7611fc-a5f1-43fd-9bff-1ccbb81666ab None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.791816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.792509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.792797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.793072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.793754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:53.794056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.795476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.810317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 69/133] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:26:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:53.816903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:53.827115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.836443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3e8b-8c8e-4bcf-a43c-fcb4f73e9e74 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:53.876494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:53.877847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:53.878131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:53.878407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] PATH_INFO: `/v3/projects/0484384e3bc142bebbd15edb6edf51a2/users/57a045168aa44e0386b65064bd0ea153/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:53.880425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] RBAC: Authorizing `identity:create_grant(project_id=0484384e3bc142bebbd15edb6edf51a2, user_id=57a045168aa44e0386b65064bd0ea153, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:53.882013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:53.900909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 65/134] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:26:53 2022] PUT /identity/v3/projects/0484384e3bc142bebbd15edb6edf51a2/users/57a045168aa44e0386b65064bd0ea153/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:53.937288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:53.976989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7dd14-8907-4938-977c-e165a363eed8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:54.015962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:54.016719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:54.017002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:54.017281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:54.019136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:54.020663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:54.022405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 70/135] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:53 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:54.027947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:54.151912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1991-64c1-4dab-92e9-aa2b33d1312d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:54.197203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:54.197911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:54.198194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:54.198513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:54.199754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:54.201223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290dcc-35ed-4edd-9432-e6db698790c9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:54.202881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 66/136] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:54 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:54.209891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:54.212077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f72e3-668c-46ee-abc7-052d20bd0858 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:54.212360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f72e3-668c-46ee-abc7-052d20bd0858 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:54.212641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f72e3-668c-46ee-abc7-052d20bd0858 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:54.214289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 67/137] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:54.224804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:54.225102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:54.225363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:54.260819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:54.262082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:54.292967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:54.395819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 68/138] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:54.413657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38c0e9-c6e1-4eca-a4d5-e7f6dd422d78 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:54.468316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:54.468984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:54.469266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:54.469540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/57a045168aa44e0386b65064bd0ea153/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:54.475439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, user_id=57a045168aa44e0386b65064bd0ea153, role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:54.476895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:54.479494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:54.509705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 71/139] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:26:54 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/57a045168aa44e0386b65064bd0ea153/roles/428b94d9bca848c1a1180b457c176272 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:54.839209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:54.967301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba05dc-67c1-4357-ab67-3140b53fe11b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:55.008527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:55.009181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:55.009467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:55.009746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:55.010894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:55.012354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28dc2e-5898-4c51-a569-99e6c043bd70 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:55.013935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 69/140] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:55.019469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:55.143334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ce4e-56c9-4392-9722-1c01d49d6a21 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:55.181025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:55.181641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:55.181919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:55.182201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:55.182898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:55.184354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca5614-c9c8-45fa-b5a8-3e818e0d5d2f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:55.225536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 72/141] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:26:55 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:26:56.307028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dadd7b-2495-4641-865a-b152b347c34e None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.307554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dadd7b-2495-4641-865a-b152b347c34e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.307630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dadd7b-2495-4641-865a-b152b347c34e None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.308935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 70/142] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:56.316573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec2aa9-16f5-496f-a447-d3f0051463f8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.317017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec2aa9-16f5-496f-a447-d3f0051463f8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.317113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec2aa9-16f5-496f-a447-d3f0051463f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.340013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3cec2aa9-16f5-496f-a447-d3f0051463f8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:56.341218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3cec2aa9-16f5-496f-a447-d3f0051463f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:56.403811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:56.407625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957053-7585-49ab-b4f6-979fb9dd9705 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.408858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957053-7585-49ab-b4f6-979fb9dd9705 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.408858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957053-7585-49ab-b4f6-979fb9dd9705 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.410121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 71/143] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:56.415500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5f42-c3b9-49f8-a5e4-b79b16ea5a56 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.415785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5f42-c3b9-49f8-a5e4-b79b16ea5a56 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.416062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5f42-c3b9-49f8-a5e4-b79b16ea5a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.446727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9d4d5f42-c3b9-49f8-a5e4-b79b16ea5a56 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:56.447938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9d4d5f42-c3b9-49f8-a5e4-b79b16ea5a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:56.465150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 73/144] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:56.475096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:56.538452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 72/145] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:26:56.576674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192b761-dc8f-4c8d-a2d4-78b505d103cc None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.576945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192b761-dc8f-4c8d-a2d4-78b505d103cc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.577158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192b761-dc8f-4c8d-a2d4-78b505d103cc None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.578839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 74/146] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:56.585687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04def7-4d0b-4e04-8bd9-4a71692374c1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.586030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04def7-4d0b-4e04-8bd9-4a71692374c1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.586310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04def7-4d0b-4e04-8bd9-4a71692374c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.607738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7b04def7-4d0b-4e04-8bd9-4a71692374c1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:56.608945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7b04def7-4d0b-4e04-8bd9-4a71692374c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:56.688518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 73/147] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:56.724414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c283-9bc6-4af6-bb69-95c848d7b3cb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.725025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c283-9bc6-4af6-bb69-95c848d7b3cb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.725493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c283-9bc6-4af6-bb69-95c848d7b3cb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.727401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 75/148] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:56.733032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:56.733575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:56.734070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:56.759691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:56.761208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:56.848452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 74/149] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:56.957608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186c283-9bc6-4af6-bb69-95c848d7b3cb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.038602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.053276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:57.113022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186c283-9bc6-4af6-bb69-95c848d7b3cb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.166274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.167386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.167983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.168498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.186913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:57.192561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:57.193391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:26:57.194127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:57.196487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 76/150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:57.202255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.205199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c94c5-23a4-430f-911a-43abf05d2725 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.260598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.261678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.262133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.262603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.268244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:57.285542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:57.292284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:57.293275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:57.294031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:57.294953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 75/151] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:57.300001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.382137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1ae1-29cf-49b0-8534-9f9c92d51d35 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.410120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:57.445320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae2fea-c781-4f6b-8be4-2da673b7a996 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.466130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.466787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.467074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.467352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.468044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:57.468331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:57.469729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:57.489047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.489669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.489946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.490219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.490919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:57.491221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:57.492607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:57.493528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 77/152] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:57.494811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.502763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 76/153] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:26:57.504011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.621845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0e5a-6929-4a30-8e95-421216df8b89 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.624403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:57.629734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13e082-bc7f-4eab-917a-92d8555cdc52 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.670986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.671553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.671839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.672118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.683292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:57.691361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.692700 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.692983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.693261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.695026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3f4dec96-209b-4588-a43d-79e01a472d8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:57.704562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:57.705919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:26:57.706673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:26:57.708148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 78/154] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:57.708361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d627-68a9-4c72-a562-eeb806eb3d77 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.711534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d627-68a9-4c72-a562-eeb806eb3d77 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.711534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d627-68a9-4c72-a562-eeb806eb3d77 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.711534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 79/155] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:57.712742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3d627-68a9-4c72-a562-eeb806eb3d77 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.716095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:57.722734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:57.723861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:26:57.724831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:57.730804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 77/156] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:57.737367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:57.878191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8595ac-2de1-4309-9676-c330f6086024 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.936145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3d627-68a9-4c72-a562-eeb806eb3d77 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:57.946441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:57.947113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:57.947393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:57.947670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:57.953314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:57.954811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:57.957350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 78/157] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:57.958369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.012851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.015204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.015653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.016083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] PATH_INFO: `/v3/users/29a8ffd181d848dcb0dc05853003dc6d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.019962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] RBAC: Authorizing `identity:get_user(user_id=29a8ffd181d848dcb0dc05853003dc6d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.021588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b7b43-801a-4916-acc7-9d946842a176 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.023629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 80/158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/users/29a8ffd181d848dcb0dc05853003dc6d => generated 289 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.024881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.027097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.028831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.053038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.066449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:58.073966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:58.119735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.124624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a882e9-9d97-4fcc-b228-7c94cabb1cf4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.183939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.184836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.185274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.185706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.186601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:58.187086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.189863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.199040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 79/159] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:26:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.200420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.242356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 81/160] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:58.243578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.312488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bae057-21b9-4fa0-84ed-677d08280d69 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.340751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.354575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.355530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.355993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.356458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.357537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:58.358135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.359789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.370416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 80/161] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.371556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.552771 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431e89-c7a0-4fa8-95c8-f47eac1c6b44 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.604731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.605655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.606036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.606406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.608104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.609735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.612072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 81/162] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.613113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.648608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69e6ad-5ea7-44ba-a826-efab3ef74508 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.670843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.726672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cad87-fe8f-4c13-bdc2-c19df5a0833b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.763151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.763809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.764096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.764374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] PATH_INFO: `/v3/users/11b0f239f5044e6880117c0daf3fe327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.768032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] RBAC: Authorizing `identity:get_user(user_id=11b0f239f5044e6880117c0daf3fe327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.768931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.769553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.769835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.770105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.770788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.772991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 82/163] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/users/11b0f239f5044e6880117c0daf3fe327 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.774066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.823119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.889393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.891115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.896558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396439c-a37a-44a1-8e7f-cb463ff75930 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:58.916275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:58.937293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:58.938395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:58.939174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:58.940187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:58.941729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:58.943401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:58.945377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 83/164] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:58.946535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:58.995257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 82/165] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:58.996115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.057059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.071269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43702142-5d34-4c97-845e-b696ab4dbbfe None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.099727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de839fca-7338-4d8e-879f-d8b3d8eb46b8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.110134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.111744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.112222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.112785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.124620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:26:59.131198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:26:59.132095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:26:59.132888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:26:59.135833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 84/166] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:26:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:26:59.138077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.138487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.142277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.142277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.142277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/29a8ffd181d848dcb0dc05853003dc6d/roles/a62c9a189f5941a393d9ba582ec6ea49` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.144027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, user_id=29a8ffd181d848dcb0dc05853003dc6d, role_id=a62c9a189f5941a393d9ba582ec6ea49)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.145474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d4649-961d-42aa-9a87-afa63551b561 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.165980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 83/167] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:26:58 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/users/29a8ffd181d848dcb0dc05853003dc6d/roles/a62c9a189f5941a393d9ba582ec6ea49 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:26:59.176230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004944e-0457-4d2d-87e3-ab2c42678991 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.176562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004944e-0457-4d2d-87e3-ab2c42678991 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.176789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004944e-0457-4d2d-87e3-ab2c42678991 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.178242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 84/168] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:59.182577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004944e-0457-4d2d-87e3-ab2c42678991 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.207039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.275484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45a505-7b55-4dd8-aaac-077020fe78d0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.318598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.319744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.320241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.320726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.401984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.403898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.411440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004944e-0457-4d2d-87e3-ab2c42678991 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.441206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.457840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.462991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.463875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.464449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.464925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.516405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 85/169] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:59.518373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.549915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.551472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.631683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b9c0-4c2f-4098-b574-4dfb2f4d9575 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.669839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.670403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.670681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.670993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.671153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 85/170] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:26:59.672179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.672644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.673698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f2e8-8209-40f6-ae2e-c0a7ede122d2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.675884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 86/171] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:59.677267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1870-4159-4dc5-b7cd-81db44bc4b49 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.677553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1870-4159-4dc5-b7cd-81db44bc4b49 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.677825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1870-4159-4dc5-b7cd-81db44bc4b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.698736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-125e1870-4159-4dc5-b7cd-81db44bc4b49 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:26:59.699928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-125e1870-4159-4dc5-b7cd-81db44bc4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:26:59.700063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.710082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.776230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 87/172] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:26:59.777862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb11d90-cc7b-480b-beab-68165dde52f3 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.778138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb11d90-cc7b-480b-beab-68165dde52f3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.778406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb11d90-cc7b-480b-beab-68165dde52f3 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.779846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 88/173] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:59.784935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb11d90-cc7b-480b-beab-68165dde52f3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.801277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaf61-5659-4936-ac7b-a94362b5bcdc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.838413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.839381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.839853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.840336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.841319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:26:59.841706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.843188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad85a8-f5f3-46cd-a1a4-fbd4b2c50d5b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.852682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 86/174] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:59.854139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306738b-9c4f-4312-ac7a-c00a3c70b8c8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.854635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306738b-9c4f-4312-ac7a-c00a3c70b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.855137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306738b-9c4f-4312-ac7a-c00a3c70b8c8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.856662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 87/175] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:26:59.857546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306738b-9c4f-4312-ac7a-c00a3c70b8c8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.918320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb11d90-cc7b-480b-beab-68165dde52f3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.948594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:26:59.968304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:26:59.969373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:26:59.969974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:26:59.971102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:26:59.971921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306738b-9c4f-4312-ac7a-c00a3c70b8c8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:26:59.972345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:26:59.974046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:26:59.981343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 89/176] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:26:59.981343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:26:59.996349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.014905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.015805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.016296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.016762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] PATH_INFO: `/v3/projects/0484384e3bc142bebbd15edb6edf51a2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.018604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] RBAC: Authorizing `identity:get_project(project_id=0484384e3bc142bebbd15edb6edf51a2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.020280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d9d30-0d45-41d7-b5c9-3c66825b38d1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.022105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 88/177] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/projects/0484384e3bc142bebbd15edb6edf51a2 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:00.023430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.023934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.024266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.042370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:00.043662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:00.112514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb480f08-7e74-458d-8e1b-63d1a49932c4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.113729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 89/178] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:00.120181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.154007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.155276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.156003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.156490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.158616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:00.159461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.161326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e51a-2bda-4cb4-a936-10f27a30d6f9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.168950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 90/179] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:26:59 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:00.171023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.171536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.172331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.195416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:00.196705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:00.221208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.246073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65340f33-ad96-483b-aaa1-2a2f69d73eba None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.270874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 91/180] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:00.272026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.286247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.287985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.288882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.289379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.289868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] PATH_INFO: `/v3/users/57a045168aa44e0386b65064bd0ea153` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.291303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] RBAC: Authorizing `identity:get_user(user_id=57a045168aa44e0386b65064bd0ea153)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.293281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.295474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 90/181] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/users/57a045168aa44e0386b65064bd0ea153 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:00.302148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.405252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bae31-53a3-4a93-a239-269960982e40 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.430735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67b6ac-dc07-486c-9113-3d32bf187cea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.444625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.445260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.445564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.445824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.446558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=11b0f239f5044e6880117c0daf3fe327) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:00.446913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.448441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.456553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 92/182] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/role_assignments?user.id=11b0f239f5044e6880117c0daf3fe327&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:00.457591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.473222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.474233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.474758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.475255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.476210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=0484384e3bc142bebbd15edb6edf51a2, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:00.476735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.478420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.486801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 91/183] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=0484384e3bc142bebbd15edb6edf51a2&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:00.488181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.507080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.541842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.586187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259bc46-ec54-4ae2-8c17-7fb9457c6d19 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.622441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d28584-7596-4f88-a901-ed7868d959c0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.647566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.648841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.649274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.649700 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.664628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:00.666616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.667463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.667845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.668217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:00.675357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:00.676501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:00.677411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:00.680543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 93/184] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:00.682029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.792355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:00.793149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:00.813395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.857869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96889e-a6d6-4b72-83fd-6edfa747e1c3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:00.896197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:00.903187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:00.904035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:00.904366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:00.904668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:00.918182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:00.924309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:00.925195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:00.925979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 94/185] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:00.926747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:00.958864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 92/186] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:00.960247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.043161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444916f-2b03-4d51-a6d0-e81e33e661a6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.069657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54b25-ff7e-443f-8ae8-14724d1e79eb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.080131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.081058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.081343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.081822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.082440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:01.082769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.083381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.084413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.093743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 95/187] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.099402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.106084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.106723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.107007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.107279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.108399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.109841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.112031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 93/188] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.112780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.158640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.210419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f68f6-2554-4398-b259-ad7313b808ca None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.223187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42780cd-cad9-4d9c-a901-7e83714f9c9e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.247392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.247988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.248268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.248546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.249210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:01.249515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.250934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.263733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 96/189] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.263896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.264002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.264083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.264083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.264083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.265244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.266679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.268607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 94/190] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.273338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.325176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.367680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa2d6-6925-4637-b4c8-559904f5c4e9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.379984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aceff9-6fdc-4dc6-b5fb-3d1e2295458f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.382227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.403537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.404308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.404694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.405067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.415410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.416179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.416463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.416736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.418075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.419691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.419836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.421129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.427993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 97/191] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.427993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.450734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 95/192] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:01 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:01.450734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.568565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80d729-e648-4de5-91f8-0a83f901af6f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.569013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c28d54-73ac-418a-9e8d-31afa6a7f3e0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.572440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.607407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.608197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.608479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.608747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.609882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.611803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.613044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 98/193] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.614226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.620916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.622186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.622186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.622186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.624007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=57a045168aa44e0386b65064bd0ea153) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:01.624007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.626637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8b78-f4e5-42d1-8502-9e9f52a89df6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.635978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 96/194] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/role_assignments?user.id=57a045168aa44e0386b65064bd0ea153&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=428b94d9bca848c1a1180b457c176272 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:01.636676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f9f9-e49e-41f9-a799-757b891dba1f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.636957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f9f9-e49e-41f9-a799-757b891dba1f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.637286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f9f9-e49e-41f9-a799-757b891dba1f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.638625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 97/195] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:01.644057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f9f9-e49e-41f9-a799-757b891dba1f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:01.662369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.739268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22243ae-7335-4577-ad68-2463c1133367 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.790247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.791123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.791404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.791678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.793539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f9f9-e49e-41f9-a799-757b891dba1f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:01.841683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:01.842843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:01.843211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:01.843552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:01.883297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.884767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112aa661-0079-4de6-a989-78d9aff9eba4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:01.895225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.937757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:01.945413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:01.947790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.015579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 99/196] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:02.016954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.022665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.023063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.043211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:02.044626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:02.075731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 98/197] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:02.076676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.132369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 100/198] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:02.133907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.171063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:02.232659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:02.260640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17774a5b-519b-4375-bf4e-4dc0765b9dc6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.267721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb887cd-8835-4f6c-a811-ea39e4670c03 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.309442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.310084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.310360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.310633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.315584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:02.317034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.318862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 101/199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:02.325372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.344343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.344978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.345254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.346062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.349970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:02.351453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.359045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 99/200] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:02.359892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.483457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31f2e6-2f4e-4313-a0cf-8fb08b2b5537 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.492236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2778b15-1e0f-44da-9664-fc1b7d2dd95f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.533462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.533997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.534190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.534560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.534925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.535361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.535797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=c301620c4b5b4077b905e2a62657dd77) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:02.535964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.536218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:02.536582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.537467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=c301620c4b5b4077b905e2a62657dd77) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:02.537825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.538161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:02.540012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.543840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 102/201] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/role_assignments?user.id=c301620c4b5b4077b905e2a62657dd77&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=428b94d9bca848c1a1180b457c176272 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:02.544429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:02.544705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.548000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:02.549593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 100/202] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/role_assignments?user.id=c301620c4b5b4077b905e2a62657dd77&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:02.681923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fad2-2073-4102-af99-2ec879f7f64e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.722443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.723204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.723536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.723811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.736851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:02.742264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:02.743194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:02.744607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 103/203] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:02.751659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:02.891637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a341-f628-4ce2-97c5-c80eae48184c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:02.937214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:02.937837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:02.938116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:02.938391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:02.939385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:02.939687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:02.941074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:02.952178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 101/204] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:02.958787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:03.026084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:03.068451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bcc8-4ae4-478d-b20d-ea187a276e5a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:03.103205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:03.103935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.104217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.104483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] PATH_INFO: `/v3/users/29a8ffd181d848dcb0dc05853003dc6d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.105733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] RBAC: Authorizing `identity:get_user(user_id=29a8ffd181d848dcb0dc05853003dc6d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:03.107304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:03.109253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 104/205] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:02 2022] GET /identity/v3/users/29a8ffd181d848dcb0dc05853003dc6d => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:03.115493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:03.203597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:03.253121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f2ee6-36d7-4395-a1a7-2dea25b499f9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:03.290351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:03.291378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.291896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.292375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.362660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:03.364563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:03.435137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:03.468101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 102/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:03.473880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:03.603839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e98c-60d1-45ae-8d64-7e59b4c4516e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:03.642922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:03.650452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:03.651118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.651393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.651660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.652852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:03.654391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f1326-2621-49f9-bc2a-a469495466ee None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:03.656340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 105/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:03 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:03.662785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:03.763491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e41ca-c38f-4bf8-a197-a6257906c924 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:03.798888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:03.799497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.799771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.800040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.800766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, scope.project.id=f1260705bec44aa09dcd7d3d097b4913, user.id=29a8ffd181d848dcb0dc05853003dc6d) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:03.801066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:03.802494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3c81-a056-4691-bd9d-b5c2481b098e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:03.808631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 103/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:03 2022] GET /identity/v3/role_assignments?user.id=29a8ffd181d848dcb0dc05853003dc6d&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:03.865889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac32d4-f0d4-4f46-8837-921fc2bdc0ec None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.866405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac32d4-f0d4-4f46-8837-921fc2bdc0ec None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.866405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac32d4-f0d4-4f46-8837-921fc2bdc0ec None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.867790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 106/209] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:03.873433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f2d10-5726-4ab0-9f01-f2b9804d67d9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:03.873778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f2d10-5726-4ab0-9f01-f2b9804d67d9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:03.874055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f2d10-5726-4ab0-9f01-f2b9804d67d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:03.893232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c14f2d10-5726-4ab0-9f01-f2b9804d67d9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:03.894486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c14f2d10-5726-4ab0-9f01-f2b9804d67d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:03.917451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:03.971666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 104/210] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:04.081180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f150-b1bf-4f3e-a091-14fdb644f624 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.081504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f150-b1bf-4f3e-a091-14fdb644f624 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.081727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f150-b1bf-4f3e-a091-14fdb644f624 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.083114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 107/211] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:04.089302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.089585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.089861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.112208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:04.113410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:04.195627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 105/212] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:04.290256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2f150-b1bf-4f3e-a091-14fdb644f624 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.408765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2f150-b1bf-4f3e-a091-14fdb644f624 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.444266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:04.445086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.445369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.445694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.456773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:04.462167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:04.463054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:04.463819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:04.468354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 108/213] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:04.469723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.526281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.571229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:04.579379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6594cd-d8dc-4756-9743-cdd4a28456b4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.602343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:04.633274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:04.634171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.634518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.634896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.635948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:04.636324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:04.637780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:04.649469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 106/214] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:04.654089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70bcf4-7f2b-4597-9453-524bcb71dc59 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.655650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.696657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:04.697215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.697497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.697771 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.698953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:04.700411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:04.707546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 109/215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:04.709182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.766280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876356e-025e-4663-8151-7ad0f2260bc9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.803968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:04.804935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.805486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.805934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] PATH_INFO: `/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.812527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf857-9da4-434f-acf7-fe598faf1a78 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.821618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a39ce102ed4909a7ff2fec7b0c1665)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:04.823478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:04.831071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 107/216] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:04 2022] GET /identity/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:04.832569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:04.840716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:04.849908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:04.850614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.850982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.851330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.852054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:04.853696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbbf74-e8fb-40ad-9dc1-d71fa949cb89 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:04.871083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:04.887541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 110/217] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:04 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:04.948997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6adc7b-bb42-4786-bb2e-1240c5a5e711 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.949305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6adc7b-bb42-4786-bb2e-1240c5a5e711 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.949565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6adc7b-bb42-4786-bb2e-1240c5a5e711 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.951401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 111/218] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:04.958915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd7f9-eec5-42df-a51a-23a7f26c6b86 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:04.960121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd7f9-eec5-42df-a51a-23a7f26c6b86 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:04.960487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd7f9-eec5-42df-a51a-23a7f26c6b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:04.976994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ab62-8c21-4a84-a5b7-e4739c81d771 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:04.987310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e3dcd7f9-eec5-42df-a51a-23a7f26c6b86 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:04.989107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e3dcd7f9-eec5-42df-a51a-23a7f26c6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:05.017720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.018340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.018620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.018937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.078751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 112/219] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:05.080583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d039a1-6475-4a8e-8208-9c4ebe8a4653 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.080869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d039a1-6475-4a8e-8208-9c4ebe8a4653 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.081519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d039a1-6475-4a8e-8208-9c4ebe8a4653 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.082569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 113/220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:05.088374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.088747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.089366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.103321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:05.105524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:05.111547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:05.112929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:05.119548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:05.202250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 114/221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:05.236689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 108/222] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:05.242506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.293391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:05.358678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91168-208e-42fe-84b6-81f235e6edea None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:05.365994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.395866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.396497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.396771 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.397036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.398280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:05.399867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:05.401832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 115/223] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:05.407338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.471224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6961fc-aa64-49b9-a19b-1d81e1850b81 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:05.479426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:05.511644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.513112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.513398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.513727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.526070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:05.532264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:05.533685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:05.534450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:05.535052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 109/224] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:05.539548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.551572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee350211-7a23-4d26-a608-647e9f2bfe73 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:05.566516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:05.605457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.606804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.607263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.607959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.608846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=a3a39ce102ed4909a7ff2fec7b0c1665) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:05.609459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:05.611834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:05.619327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 116/225] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/role_assignments?user.id=a3a39ce102ed4909a7ff2fec7b0c1665&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:05.620302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.670083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae3cf-a342-4473-b789-ee92ca16f47d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:05.745776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.748077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.749805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.750242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.772432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:05.777783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:05.778922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:05.779907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:05.786005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 110/226] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:05.791773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.806249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c39a-46a5-4e88-b626-17757e749a3f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:05.844444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:05.850443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:05.851334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:05.852466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:05.853564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:05.854427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:05.855118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:05.858672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:05.872842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 117/227] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:05.879763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:05.939869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:06.004938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2d859-6a63-4ef0-915d-f47fe04a8a58 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.032459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99679-c46b-48d1-9c03-ca0310700dfe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.072718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.073597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.073881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.074152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.079395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.082052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.084817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 118/228] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:06.085676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.086529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.087060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.087568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.088438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:06.088952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.090520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.091531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:06.111498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 111/229] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:06.116999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:06.255244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc387c-bdad-4d96-a4b3-7bc370d18e49 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.259617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:06.293954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:06.314288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbedc3d-181c-4627-af54-2b84a2f5aa67 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.321446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.322082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.322354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.322629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.325311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.326781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.328408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 119/230] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:06 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:06.334682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:06.375784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.376449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.376729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.376998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.378844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.380295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.383545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 112/231] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:06 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:06.387360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:06.513262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ed819-e5e5-445b-9552-7129664a0380 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.566667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0be239-3e30-44fe-b6dd-3cd0a6898dda None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.576315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.576987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.577264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.577536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/users/c301620c4b5b4077b905e2a62657dd77/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.585086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, user_id=c301620c4b5b4077b905e2a62657dd77, role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.586938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.615831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 120/232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:06 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/users/c301620c4b5b4077b905e2a62657dd77/roles/428b94d9bca848c1a1180b457c176272 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:06.638273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.639244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.639534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.639814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.641088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.642644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2aa9b-d01c-4218-a6e2-902546160f91 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.644750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 113/233] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:06 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:06.650674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:06.697770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:06.791220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1530f-02ac-48c7-9818-47e14415dd7f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:06.829155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:06.830389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:06.830674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:06.831003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/users/c301620c4b5b4077b905e2a62657dd77/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:06.833121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, user_id=c301620c4b5b4077b905e2a62657dd77, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:06.834586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81232c2-5aba-4a27-a6bc-b6d2f896a433 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:06.861201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 121/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:06 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/users/c301620c4b5b4077b905e2a62657dd77/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:07.486961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a95f-8e17-4b3f-9389-5ccb63df935c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.487473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a95f-8e17-4b3f-9389-5ccb63df935c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.487554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a95f-8e17-4b3f-9389-5ccb63df935c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.489120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 114/235] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:07.494520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69652767-e714-4ce8-bfed-0a1e286ff605 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.494962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69652767-e714-4ce8-bfed-0a1e286ff605 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.495153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69652767-e714-4ce8-bfed-0a1e286ff605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.518453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-69652767-e714-4ce8-bfed-0a1e286ff605 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:07.519696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-69652767-e714-4ce8-bfed-0a1e286ff605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:07.610592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6935a3-54a6-4503-a4d4-e2fc8ceb4821 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.611483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6935a3-54a6-4503-a4d4-e2fc8ceb4821 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.611765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6935a3-54a6-4503-a4d4-e2fc8ceb4821 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.613362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 115/236] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:07.618993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7fd1-23e2-4014-8ddf-2e17c03276cc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.619283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7fd1-23e2-4014-8ddf-2e17c03276cc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.619722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7fd1-23e2-4014-8ddf-2e17c03276cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.628782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 122/237] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:07.651782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d73b7fd1-23e2-4014-8ddf-2e17c03276cc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:07.653204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d73b7fd1-23e2-4014-8ddf-2e17c03276cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:07.679640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:07.748107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 116/238] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:07.802553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09951f-aec5-4af7-b437-b6a7b259044f None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.803014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09951f-aec5-4af7-b437-b6a7b259044f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.803418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09951f-aec5-4af7-b437-b6a7b259044f None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.805110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 123/239] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:07.810319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:07.810755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:07.811134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:07.831004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:07.832178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:07.916017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 117/240] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:08.098517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09951f-aec5-4af7-b437-b6a7b259044f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.124117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.145317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.212891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09951f-aec5-4af7-b437-b6a7b259044f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.250487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.251158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.251988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.252260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.253645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:08.255276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:08.257322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 124/241] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:08.265170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.309431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.332885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627753-6e0f-478b-bfab-6ade5fc0b657 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.383016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.383635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.383919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.384204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.384888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.396805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67d289-119f-4a77-84c6-071921ad4d6b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.400717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:08.406283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:08.407181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:08.408506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 118/242] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:08.409655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.443350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.444385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.444858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.445352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.446230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:08.447885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:08.472542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 125/243] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:08 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:08.473914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.545394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740c92-240a-4edf-891c-1d6a355d533c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.629477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.630132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.630415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.630704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.654816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:08.660177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:08.661051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:08.661856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:08.662455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 119/244] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:08.669068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.675120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.695912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabd7c-2615-4076-95ee-ee2d078039b1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.732480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.733104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.733612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.733716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.734314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:08.734617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:08.737515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:08.746810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 126/245] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:08.752970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.766538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.801418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e66b6-5f99-4376-816f-8620495723ee None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.854884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.855762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.856142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.856540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.857406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:08.857720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:08.859150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd1d31-bf53-4563-82e5-fa0e2643c301 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:08.870217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 120/246] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:08.873340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9307aa6-7465-4747-b9e1-766f00b8b612 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.873629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9307aa6-7465-4747-b9e1-766f00b8b612 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.873904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9307aa6-7465-4747-b9e1-766f00b8b612 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.875577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 121/247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:08.877787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9307aa6-7465-4747-b9e1-766f00b8b612 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:08.898304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d32b18-8860-4e86-ac69-072026faa154 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:08.947550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:08.948652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.949126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.949574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] PATH_INFO: `/v3/users/a00087e9726c4e74bc50d1a005e7a8c8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.967881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] RBAC: Authorizing `identity:get_user(user_id=a00087e9726c4e74bc50d1a005e7a8c8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:08.969367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3729be8-f684-41b3-9cd5-176a6cb6526e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:08.971717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 127/248] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/users/a00087e9726c4e74bc50d1a005e7a8c8 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:08.972905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:08.973182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:08.973451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:08.979791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:08.995846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:08.997297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:09.028466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9307aa6-7465-4747-b9e1-766f00b8b612 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.061996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:09.071094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:09.071742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.072025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.072299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] PATH_INFO: `/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.076435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a39ce102ed4909a7ff2fec7b0c1665)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:09.076435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:09.076641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 122/250] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:08 2022] GET /identity/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:09.077030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 128/250] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:09.077270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.083965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.216670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2a3e7-f480-4ff1-a7a8-5350a668d467 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.260591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:09.261355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.261636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.261904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.275018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:09.364946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8436f-651b-48be-9d52-00a874685a05 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.383298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:09.385294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:09.482433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:09.483356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.483782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.484221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.485533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:09.487238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60530a1-85bf-49fe-9c5f-f505c04c992b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:09.489391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 129/251] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:09 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:09.490729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b4c5-4101-4f0f-86a5-df182c15ab5a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.491159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b4c5-4101-4f0f-86a5-df182c15ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.491640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b4c5-4101-4f0f-86a5-df182c15ab5a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.493308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 130/252] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:09.494225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b4c5-4101-4f0f-86a5-df182c15ab5a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.509187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 123/253] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:09.510259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.546274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:09.585157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:09.624977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1b98-bce1-4078-b9e7-c898b00865ec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.637925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b4c5-4101-4f0f-86a5-df182c15ab5a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.663158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:09.664306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.664761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.665209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/users/a3a39ce102ed4909a7ff2fec7b0c1665/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.667761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, user_id=a3a39ce102ed4909a7ff2fec7b0c1665, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:09.669531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d2bc9-4713-4e1e-a765-b04d93c8409b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:09.680491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:09.681180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.681459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.681733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.686454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 124/254] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:09 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/users/a3a39ce102ed4909a7ff2fec7b0c1665/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:09.688263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:09.689606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:09.690145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:09.707371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:09.712457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:09.712726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:09.713618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 131/255] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:09.720741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.722566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:09.855106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 125/256] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:09.856011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:09.909722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:09.937314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f5029-59dc-414c-a7e5-b8f166658f08 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:09.997939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:10.020521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.021160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.021433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.021701 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.024181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:10.026466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:10.028150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 132/257] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:09 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:10.034094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.068273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098970fd-8e3c-4daf-8043-a8400496f53d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.156108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.157075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.157589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.158056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.159110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:10.159628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:10.161358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:10.170743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 126/258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:10.170743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.218061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b1c57-ca6f-4e16-bd08-1c29d0a09771 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.300410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.301084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.301363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.301630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.302435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=a00087e9726c4e74bc50d1a005e7a8c8) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:10.302764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:10.306107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:10.320750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 133/259] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/role_assignments?user.id=a00087e9726c4e74bc50d1a005e7a8c8&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:10.321633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.327090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:10.365055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244781-7d7e-4119-8586-08bf9afdcf42 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.440854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:10.477407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.478062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.478343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.478618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.503543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:10.515962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f943-bda7-4db6-80c3-4db4c0de5f35 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:10.516915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:10.517796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:10.518649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:10.522896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 127/260] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:10.530428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.570386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.571436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.571915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.572436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.574007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:10.575719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:10.577671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 134/261] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:10.583779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.760693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2157-4eaf-4b4b-84e7-6deb815b2472 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.801359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e28a-6243-4f93-b57c-e6f417e246ba None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:10.843435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:10.851956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.852584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.852861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.853132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.853804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:10.854101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:10.855567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:10.867680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:10.868310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:10.868594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:10.868873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:10.874093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 128/262] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:10.880017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:10.899671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:10.998368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.000180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422e7bc-80fe-4ad5-814a-a034803ee670 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.126983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 135/263] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:27:11.128555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2f13d-8576-4651-bf8f-d4ab042eaaa1 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.128863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2f13d-8576-4651-bf8f-d4ab042eaaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.129162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2f13d-8576-4651-bf8f-d4ab042eaaa1 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.130798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 136/264] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:11.132575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2f13d-8576-4651-bf8f-d4ab042eaaa1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.140032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70534-1578-40c4-aa8b-ad6a35d04cfb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:11.183265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:11.207894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:11.208535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.208815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.209089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] PATH_INFO: `/v3/users/c301620c4b5b4077b905e2a62657dd77` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.215424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] RBAC: Authorizing `identity:get_user(user_id=c301620c4b5b4077b905e2a62657dd77)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.216991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8b732-9924-48c7-b073-cd6c735e9019 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.219010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 129/265] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:10 2022] GET /identity/v3/users/c301620c4b5b4077b905e2a62657dd77 => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:11.220214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.220609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.220886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.245911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2f13d-8576-4651-bf8f-d4ab042eaaa1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:11.249533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:11.250827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:11.284843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:11.285582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.285911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.286189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.287372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.288956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.290924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 137/266] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:11.295585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.345011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 130/267] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:11.345888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.363769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:11.399649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:11.408546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6635d7-22f1-4e93-9bf4-834949753f53 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:11.457922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:11.458559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.459703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.459982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.554653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.556318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232a0e6-8a2e-47e1-bd3a-9f45f7b305af None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.581542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d6f96-6a5b-47e9-9ebb-e9792feeb15d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:11.660058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:11.670647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 138/268] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:11.671796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c4d2-6428-4ca0-93c3-fc352582c9fb None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.672172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c4d2-6428-4ca0-93c3-fc352582c9fb None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.672363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c4d2-6428-4ca0-93c3-fc352582c9fb None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.673750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 139/269] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:11.678511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947c4d2-6428-4ca0-93c3-fc352582c9fb None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.679020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:11.679904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.680397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.680875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.681814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=c301620c4b5b4077b905e2a62657dd77) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:11.682322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.684016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f424-c77c-4eb9-bcb0-3a566c96e022 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.691747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 131/270] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/role_assignments?user.id=c301620c4b5b4077b905e2a62657dd77&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=428b94d9bca848c1a1180b457c176272 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:11.693311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.693797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.694270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.747990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:11.749691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:11.801526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947c4d2-6428-4ca0-93c3-fc352582c9fb None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:11.842847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:11.843485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:11.843762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:11.844033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:11.845376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:11.847101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:11.849943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 140/271] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:11.850789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.894729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:11.907065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 132/272] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:11.907900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:11.958189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:12.030067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a73b-801d-4a3f-97a2-2b849b9d0e08 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.078773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.080262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.080685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.081124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.082989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:12.084595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:12.086383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 141/273] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:12.092037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:12.104071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9bed7-5d7c-4224-a8d2-29f6a5142f4e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.147580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.148474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.148898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.149312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.150184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=c301620c4b5b4077b905e2a62657dd77) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:12.150646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:12.152671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:12.160837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 133/274] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:11 2022] GET /identity/v3/role_assignments?user.id=c301620c4b5b4077b905e2a62657dd77&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:12.233788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426cbf-63e3-4dbb-91eb-bd91caab8300 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.264366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:12.275001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.275834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.276261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.276692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.277494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:12.279140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:12.315968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:12.317566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 142/275] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:12 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:12.396169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:12.445139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b2cb-f6b5-4285-8104-285481293ed9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.485998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.487098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.487596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.488065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.500146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-62dec617-bc1e-4e1c-8d58-8f5340e5eab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:12.505901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:12.506799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:12.507668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:12.513861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 134/276] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:12.517295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:12.616578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65bcb1-274e-49af-b16a-dfb3c6f4ac5e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.617206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65bcb1-274e-49af-b16a-dfb3c6f4ac5e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.617896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65bcb1-274e-49af-b16a-dfb3c6f4ac5e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.621603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 135/277] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:12.626210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab76-c441-4726-b130-287f3ec32030 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.627087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab76-c441-4726-b130-287f3ec32030 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.627474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab76-c441-4726-b130-287f3ec32030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.646982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4b41ab76-c441-4726-b130-287f3ec32030 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:12.648195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4b41ab76-c441-4726-b130-287f3ec32030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:12.654060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f3d2-a51f-459c-ba62-4fa120c95448 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.700943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.701559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.701837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.702109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.702810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:12.703113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:12.704737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d6faf-5348-41ae-84e9-ffe081ebdd35 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:12.718799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 143/278] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:12.718799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95633-188e-4aea-a3f5-621b84e862ce None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.718799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95633-188e-4aea-a3f5-621b84e862ce None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.718799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95633-188e-4aea-a3f5-621b84e862ce None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.719368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 144/279] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:12.723471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed95633-188e-4aea-a3f5-621b84e862ce None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:12.746815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 136/280] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:12.748068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.748352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.748623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.770031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:12.771309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:12.781309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:12.837946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:12.838718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed95633-188e-4aea-a3f5-621b84e862ce None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:12.853279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 137/281] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:12.876704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:12.877337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:12.877614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:12.877884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] PATH_INFO: `/v3/users/910c60da5f524ee4a34c24147cf06fcc` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:12.891600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] RBAC: Authorizing `identity:get_user(user_id=910c60da5f524ee4a34c24147cf06fcc)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:12.893093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:12.895057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 145/282] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:12 2022] GET /identity/v3/users/910c60da5f524ee4a34c24147cf06fcc => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:12.903373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.057553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313df3-1ff9-48ea-914b-e42d382eea7a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.085428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.114034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.114244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.114503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.114801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.115803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, user.id=910c60da5f524ee4a34c24147cf06fcc, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:13.116123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:13.117725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:13.130474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 138/283] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 24 07:27:12 2022] GET /identity/v3/role_assignments?user.id=910c60da5f524ee4a34c24147cf06fcc&scope.system=all&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:13.222990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.246045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c0380-05bd-41f0-9b08-9241e4943e68 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.267194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:13.295671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:13.304190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.304852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.305130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.305403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.334466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:13.339430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:13.340321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:13.341112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:13.347120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 146/284] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:13.355854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.365265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28138-691b-4457-ace0-2812859c7ec9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.409505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.410210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.410490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.410787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.429221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:13.441398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:13.442280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:13.443082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:13.444865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 139/285] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:13.449194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.546585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c17e53-6b64-46b1-8078-854aa0c65925 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.613905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.614526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.614836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.615110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.615779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:13.616080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:13.622656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:13.630078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54688e5-ae16-460a-a1c7-e6d44ee99e36 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.643873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 147/286] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:13.646395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:13.649487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.704556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.705380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.705661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.705935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.706603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:13.706965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:13.707070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:13.708429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:13.725168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 140/287] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:13.730175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.847933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95095a4-1aab-4641-837a-3bb12c636667 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.858718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668392d-cead-47c4-b975-50519600fc1a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:13.892490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.893347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.893747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.894153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] PATH_INFO: `/v3/users/a00087e9726c4e74bc50d1a005e7a8c8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.895568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] RBAC: Authorizing `identity:get_user(user_id=a00087e9726c4e74bc50d1a005e7a8c8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:13.897530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:13.899925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 141/288] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/users/a00087e9726c4e74bc50d1a005e7a8c8 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:13.905845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:13.953466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:13.979422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:13.980087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:13.980366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:13.980645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] PATH_INFO: `/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:13.981827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a39ce102ed4909a7ff2fec7b0c1665)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:13.983305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:13.984913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 148/289] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/users/a3a39ce102ed4909a7ff2fec7b0c1665 => generated 297 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:13.995595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:14.036323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dfb7a-fd12-445e-bee5-6599ddb6f356 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:14.071183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:14.072421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.072920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.073391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.074734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:14.076431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:14.078602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 142/290] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:14.085824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:14.200659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:14.221493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e79b0-f1b3-433c-8428-614ef16ceef1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:14.224490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380075-243e-4719-a857-b22935e20b32 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:14.235857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:14.261220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:14.262309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.262817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.263289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/users/a00087e9726c4e74bc50d1a005e7a8c8/roles/a62c9a189f5941a393d9ba582ec6ea49` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.265410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, user_id=a00087e9726c4e74bc50d1a005e7a8c8, role_id=a62c9a189f5941a393d9ba582ec6ea49)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:14.267617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:14.282339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 143/291] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:14 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/users/a00087e9726c4e74bc50d1a005e7a8c8/roles/a62c9a189f5941a393d9ba582ec6ea49 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:14.306878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:14.307563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.307841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.308115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.407437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:14.408993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:14.546192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:14.553076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 149/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:14.558957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:14.682458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026af6f-c320-4ad8-860b-7a26b42b3e7a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:14.721923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:14.722565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.722867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.723135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.727402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:14.728886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5416fa0-8cb1-4b34-9758-b17fdd97f793 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:14.730930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 144/293] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:14 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:14.736815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:14.752334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb99-5098-4928-9ea1-235b665787c9 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.752615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb99-5098-4928-9ea1-235b665787c9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.752878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb99-5098-4928-9ea1-235b665787c9 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.754565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 145/294] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:14.760727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6168287-3df4-4a93-967e-343ede4af516 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.761012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6168287-3df4-4a93-967e-343ede4af516 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.761287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6168287-3df4-4a93-967e-343ede4af516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.791668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f6168287-3df4-4a93-967e-343ede4af516 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:14.792845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6168287-3df4-4a93-967e-343ede4af516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:14.827186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:14.899995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 146/295] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:14.921994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f19071-839a-4376-8fac-c9e4383d21ca None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:14.966030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:14.966684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:14.966984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:14.967242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:14.968031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=a3a39ce102ed4909a7ff2fec7b0c1665) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:14.968336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:14.969802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd839-57bc-403e-bf7d-22db335c148d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:14.983538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 150/296] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:14 2022] GET /identity/v3/role_assignments?user.id=a3a39ce102ed4909a7ff2fec7b0c1665&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:15.050577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f8c3-9f0b-4e4f-8367-45987eb4708e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.051275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f8c3-9f0b-4e4f-8367-45987eb4708e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.051275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f8c3-9f0b-4e4f-8367-45987eb4708e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.052884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 147/297] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:15.058654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.058979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.059256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.087372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:15.088532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:15.169557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:15.191918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 151/298] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:15.202944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6f8c3-9f0b-4e4f-8367-45987eb4708e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.308533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6f8c3-9f0b-4e4f-8367-45987eb4708e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:15.343919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:15.344466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.344765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.345534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.346638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:15.348103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63703a1-e2d2-4b6c-8f25-b58c63bb855d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:15.349948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 148/299] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:15.355196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.467686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ff92-1dbf-4e0a-9332-60ae903f389d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:15.508926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:15.509550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.509826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.510096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] PATH_INFO: `/v3/groups` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.510757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:15.512203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc29f60-0068-4222-9a65-ae8d0d26868c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:15.520241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93bfc-77ae-48e6-9c55-ee4d1979a943 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.520241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93bfc-77ae-48e6-9c55-ee4d1979a943 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.520241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93bfc-77ae-48e6-9c55-ee4d1979a943 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.520241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 149/300] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:15.522890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac93bfc-77ae-48e6-9c55-ee4d1979a943 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.537616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 152/301] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Aug 24 07:27:15 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:27:15.537616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.537616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.537616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.563187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:15.565166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:15.584102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:15.629276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:15.658368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 153/302] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:15.667981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac93bfc-77ae-48e6-9c55-ee4d1979a943 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:15.703192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:15.704141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.704932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.705201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.717126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:15.722567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:15.723469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:15.724825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 150/303] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:15.729502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.835598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6806a3-759d-4235-9b53-fafefc0fd9a5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:15.872521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:15.873490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:15.873968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:15.874440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:15.875360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:15.875860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:15.877452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:15.886267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 154/304] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:15.891734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.930933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:15.977923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:16.005436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:16.016351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d376de-c35c-4f0f-b393-9f2aae4ddfa6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.049399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc302cb-bcc7-4892-88fb-82eeb53166e2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.062233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.062909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.063178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.064263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] PATH_INFO: `/v3/users/d85233be71be4ed8a63d102b0108e8f7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.083308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] RBAC: Authorizing `identity:get_user(user_id=d85233be71be4ed8a63d102b0108e8f7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:16.084778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:16.086641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 151/305] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:15 2022] GET /identity/v3/users/d85233be71be4ed8a63d102b0108e8f7 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:16.092521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:16.099085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.099735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.100015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.100345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.111645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:16.117710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:16.118655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:16.119492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 155/306] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:16.124590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:16.206069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e455-dcc7-4dd6-abc7-6b41861c44e7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.230302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:16.233627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663d31-a390-4396-9fa3-c52df21fee4c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.244427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.245046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.245359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.245952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.246490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, user.id=d85233be71be4ed8a63d102b0108e8f7, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:16.246844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:16.248447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:16.260447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 152/307] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 24 07:27:16 2022] GET /identity/v3/role_assignments?user.id=d85233be71be4ed8a63d102b0108e8f7&scope.system=all&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:16.280108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.280739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.281022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.281296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.281984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:16.282323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:16.283907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:16.293989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 156/308] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:16.300394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:16.323095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:16.417850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df5ed-a4f7-46b4-9c2e-822085c66fb1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.452489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.453133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.453414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.453686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] PATH_INFO: `/v3/users/910c60da5f524ee4a34c24147cf06fcc` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.454867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] RBAC: Authorizing `identity:get_user(user_id=910c60da5f524ee4a34c24147cf06fcc)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:16.456362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421794c0-1253-41e8-a77a-d0f967c90305 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:16.458232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 153/309] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:16 2022] GET /identity/v3/users/910c60da5f524ee4a34c24147cf06fcc => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:16.463487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:16.546921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471cfe3-e0c9-48b2-a5e2-e25ead790748 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.547553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471cfe3-e0c9-48b2-a5e2-e25ead790748 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.548626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471cfe3-e0c9-48b2-a5e2-e25ead790748 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.550493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 154/310] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:16.555597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.555882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.556160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.570943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40eab-9f3c-4ac6-95f8-a8d538a35e80 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:16.575651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:16.576834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:16.618214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:16.619246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:16.619721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:16.620211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] PATH_INFO: `/v3/system/users/910c60da5f524ee4a34c24147cf06fcc/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:16.621974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=910c60da5f524ee4a34c24147cf06fcc, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:16.623464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:16.635067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 157/311] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Aug 24 07:27:16 2022] PUT /identity/v3/system/users/910c60da5f524ee4a34c24147cf06fcc/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:16.658969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 155/312] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:16.928240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:16.963203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:17.034131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900f16c-3dfa-497f-bea9-23d6bf60bd37 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:17.069860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:17.070886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.071386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.071852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.083215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:17.088276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:17.089146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:17.089905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:17.092296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 158/313] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:17.099609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:17.112723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:17.206083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860465c-d161-420b-a6af-9f64c14dc615 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:17.240546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:17.241157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.241435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.241714 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.242383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:17.242684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:17.244077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:17.253516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 156/314] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:17.258958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:17.354513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:17.370132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03c8f-ac07-4a09-968e-d9d0cc059114 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:17.406215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:17.406896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.407182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.408037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] PATH_INFO: `/v3/users/a00087e9726c4e74bc50d1a005e7a8c8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.409207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] RBAC: Authorizing `identity:get_user(user_id=a00087e9726c4e74bc50d1a005e7a8c8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:17.410725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15373efc-447a-4baf-93d7-86df4d2ef6db None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:17.412809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:17 2022] GET /identity/v3/users/a00087e9726c4e74bc50d1a005e7a8c8 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:17.418226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:17.500789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:17.527406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100555f-e7c1-4b11-b931-f954cfd0ef00 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:17.562001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:17.562614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.562928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.563464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.634351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:17.635938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:17.654287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f3cd-0b80-4f3d-8990-4bcada59b0ab None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.654582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f3cd-0b80-4f3d-8990-4bcada59b0ab None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.654932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f3cd-0b80-4f3d-8990-4bcada59b0ab None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.658815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 160/316] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:17.661776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.662053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.662325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.680803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:17.682095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:17.713433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:17.745398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 157/317] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:17.752047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:17.779177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 161/318] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:17.867883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89db41-6332-4648-8a5b-5acb0006bed1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:17.904455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:17.905345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:17.905791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:17.906205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:17.908209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:17.909797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:17.911936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 158/319] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:17 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:17.919826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:18.011131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:18.026826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73039a-7e25-4e96-be56-a8b1885fa99f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:18.059072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:18.064737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:18.065362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.065640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.065913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.066648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, scope.project.id=6d626a97866b42ed83934b9ad6319538, user.id=a00087e9726c4e74bc50d1a005e7a8c8) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:18.066978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:18.068422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:18.077535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 162/320] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:17 2022] GET /identity/v3/role_assignments?user.id=a00087e9726c4e74bc50d1a005e7a8c8&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:18.112655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:18.181494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4231-93af-4f89-966a-6dec7c92f4d3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:18.222282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:18.222940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.223221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.223496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.224604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:18.226097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8fe4b-88a9-49b6-bf18-10cc4cc71efa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:18.228049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 159/321] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:18.234324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:18.311958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b2ca-2d37-47c9-8cb4-ea1c4cb9924a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.312149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b2ca-2d37-47c9-8cb4-ea1c4cb9924a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.312409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b2ca-2d37-47c9-8cb4-ea1c4cb9924a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.314289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 160/322] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:18.319526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73de0-34ae-4f5f-9d95-9243cd50b720 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.319809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73de0-34ae-4f5f-9d95-9243cd50b720 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.321214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf73de0-34ae-4f5f-9d95-9243cd50b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.347053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4bf73de0-34ae-4f5f-9d95-9243cd50b720 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:18.348730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4bf73de0-34ae-4f5f-9d95-9243cd50b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:18.353790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a5bb-d894-4344-8239-bb237125070d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:18.390185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:18.390825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.391113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.391388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] PATH_INFO: `/v3/groups` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.392587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:18.394028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120688e-5de5-4d2b-9656-2e3f4f0a3498 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:18.409845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 163/323] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Aug 24 07:27:18 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:27:18.431370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 161/324] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:18.618068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f2bd-0143-48d3-bbd8-569b8cb36ac5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.618394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f2bd-0143-48d3-bbd8-569b8cb36ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.618633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f2bd-0143-48d3-bbd8-569b8cb36ac5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.620159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 164/325] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:18.625332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.625621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.625897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.645795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:18.647018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:18.699323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:18.702897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253f2bd-0143-48d3-bbd8-569b8cb36ac5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:18.715826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:18.741659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 162/326] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:18.812936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253f2bd-0143-48d3-bbd8-569b8cb36ac5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:18.853728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:18.854379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:18.854664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:18.854971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:18.867913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:18.873257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:18.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:18.874949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:18.876109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 165/327] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:18.881673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.011507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86f540-d0ca-4c86-87c0-e70b447b8c3f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.019033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.052936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.053563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.053876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.054119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.054827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:19.055133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.056524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.066411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 163/328] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:19.072217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.114617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:19.117929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:19.162897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c785d-9c33-4943-a387-6af0412b7f25 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.195102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407ead-24ea-4ccd-9479-e39690bde505 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.212088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.212721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.212993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.213256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.226503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:19.232108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:19.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:19.233729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:19.236972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 166/329] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:19.240999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.241939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.242983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.243091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.243464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] PATH_INFO: `/v3/users/d85233be71be4ed8a63d102b0108e8f7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.244919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] RBAC: Authorizing `identity:get_user(user_id=d85233be71be4ed8a63d102b0108e8f7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.246576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.249588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 164/330] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:19 2022] GET /identity/v3/users/d85233be71be4ed8a63d102b0108e8f7 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:19.254897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.352643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9c8c-ae0e-42bb-82e1-94f68a3e95ea None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.363659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:19.368451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756223-9059-471b-9c56-e53a1e60907a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.383800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:19.392006 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.392858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.393327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.393818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.394838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:19.395318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.396883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.406125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 167/331] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:19.411140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.411801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.412174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.412696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] PATH_INFO: `/v3/system/users/d85233be71be4ed8a63d102b0108e8f7/roles/a62c9a189f5941a393d9ba582ec6ea49` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.413874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.414188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d85233be71be4ed8a63d102b0108e8f7, role_id=a62c9a189f5941a393d9ba582ec6ea49)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.415795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.428245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 165/332] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Aug 24 07:27:19 2022] PUT /identity/v3/system/users/d85233be71be4ed8a63d102b0108e8f7/roles/a62c9a189f5941a393d9ba582ec6ea49 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:19.529819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4980d-5d52-4b09-a5c0-510422d19aa3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.568538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.569522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.569979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.570428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] PATH_INFO: `/v3/users/910c60da5f524ee4a34c24147cf06fcc` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.571809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] RBAC: Authorizing `identity:get_user(user_id=910c60da5f524ee4a34c24147cf06fcc)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.573496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b054544-60b0-4e7a-a6d7-c5dd1b0b37ca None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.575696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 168/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:19 2022] GET /identity/v3/users/910c60da5f524ee4a34c24147cf06fcc => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:19.586887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:19.715118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf72fc-d7de-45e3-9c2c-a582ff4212db None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:19.751603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:19.752264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:19.752540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:19.752814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:19.753586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] RBAC: Adding query filter params (role.id=c67e3b126ba2441aa233b7fc97e7fee1, user.id=910c60da5f524ee4a34c24147cf06fcc, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:19.753892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:19.755376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d0ce5-7195-4877-bbbd-7f20737c62fc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:19.768278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 166/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 24 07:27:19 2022] GET /identity/v3/role_assignments?user.id=910c60da5f524ee4a34c24147cf06fcc&scope.system=all&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:20.619674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf915-0590-4831-8236-58a797533896 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:20.620626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf915-0590-4831-8236-58a797533896 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:20.621124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf915-0590-4831-8236-58a797533896 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:20.622681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 169/335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:20.628709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2043-b845-4b92-bba6-9c50abcbad8c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:20.629525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2043-b845-4b92-bba6-9c50abcbad8c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:20.630001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2043-b845-4b92-bba6-9c50abcbad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:20.651519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ab6c2043-b845-4b92-bba6-9c50abcbad8c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:20.653068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ab6c2043-b845-4b92-bba6-9c50abcbad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:20.680214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:20.751470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 167/336] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:21.054797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3da5-6c32-4353-bb56-b4077bcd6a4a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.055485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3da5-6c32-4353-bb56-b4077bcd6a4a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.055993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3da5-6c32-4353-bb56-b4077bcd6a4a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:21.057835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 170/337] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:21.073607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.073886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.074325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:21.098805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:21.100255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:21.189695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 168/338] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:21.198576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3da5-6c32-4353-bb56-b4077bcd6a4a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:21.284190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:21.342580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3da5-6c32-4353-bb56-b4077bcd6a4a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:21.389633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:21.390278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.390556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.390854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:21.405088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:21.410492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:21.411366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:21.412569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 171/339] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:21.419355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:21.530646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:21.555124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd7de9-a261-46d8-8b1d-ee2b09236783 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:21.597051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:21.597660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.597936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.598206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:21.598895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:21.599193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:21.600557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:21.609515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 169/340] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:21.615596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:21.655074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:21.775214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea2e8d-486b-4f62-9f7f-e59c28a68e14 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:21.807931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:21.822958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:21.824306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.824306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.824306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:21.903913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df6e1-3c01-4160-a85d-649131539d65 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:21.917043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:21.918523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bbc44-2601-4c42-a448-d5c0816f580e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:21.996505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:21.997153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:21.997432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:21.997707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.016909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c10ee6ae-b312-493d-bcf4-d26064115480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:22.022220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:22.023097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:22.023888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:22.025072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 170/341] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:22.029299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520969-e2a3-4069-a23e-aa0d0e0947ed None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.029579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520969-e2a3-4069-a23e-aa0d0e0947ed None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.034934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520969-e2a3-4069-a23e-aa0d0e0947ed None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.036485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 171/342] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:22.037028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520969-e2a3-4069-a23e-aa0d0e0947ed None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.049036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.088757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 172/343] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:22.090047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.090328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.090601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.127460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:22.129903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:22.158920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.217004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520969-e2a3-4069-a23e-aa0d0e0947ed None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.274638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 173/344] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:22.274948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:22.275584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.275868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.276142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.276820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:22.277040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.277137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:22.278496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:22.288864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 172/345] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:22.294843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.427993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6c548-2dfa-4b79-83ae-2bbdd561dcee None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.447239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.471275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:22.472328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.472838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.473306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.475418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:22.476901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3a77-4814-44e0-bd4a-840506d888fc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:22.478571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 174/346] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:22.479762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d243e9f-c465-468a-8dc6-ed3906154fb7 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.480024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d243e9f-c465-468a-8dc6-ed3906154fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.480300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d243e9f-c465-468a-8dc6-ed3906154fb7 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.481499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 175/347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:22.488209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d243e9f-c465-468a-8dc6-ed3906154fb7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.504821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208ded0-501b-4bc9-b0f3-da16abfa0463 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.519062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.595132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:22.595859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.596143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.596423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.607402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d243e9f-c465-468a-8dc6-ed3906154fb7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.652109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:22.652760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.653328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.653606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.661271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:22.662778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68cf0e-d73e-4e58-bf8a-f3a56ce363c5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:22.664802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 176/348] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:22.666533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.666943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.667319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.686380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:22.687549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:22.692746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:22.694216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:22.773370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 177/349] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:22.774026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.783879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.791102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:22.838331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 173/350] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:22.839054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:22.914624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63330bc-eddd-421b-9d51-7efbf90379ae None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.957990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:22.958769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.959181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.959514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.972603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d1576-a847-40e0-94f8-463910a499e5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:22.973951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-45e5a240-4357-4002-b958-a54d5f4369fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:22.980019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:22.980910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:22.981690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 178/351] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:22.985819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad9f2-6c8f-48f9-8ddd-75da3f295fae None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:22.985819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad9f2-6c8f-48f9-8ddd-75da3f295fae None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:22.985819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad9f2-6c8f-48f9-8ddd-75da3f295fae None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:22.986360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 179/352] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:22.987067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bad9f2-6c8f-48f9-8ddd-75da3f295fae None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.014062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.014674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.014994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.015271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.016004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=f1260705bec44aa09dcd7d3d097b4913) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:23.016307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.018258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.023999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 174/353] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.024788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.066082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.085635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.130582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bad9f2-6c8f-48f9-8ddd-75da3f295fae None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.166915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f977f3-25a3-49be-abd1-b6af56739670 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.186566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.188034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.189544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.189984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.191996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.193613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3389ae4-ce79-42af-9d6a-b45ef6b05f19 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.196099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 180/354] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:22 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.197857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.198283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.198740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.212698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.213426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.213854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.214226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.215085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:23.215566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.217187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.224360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:23.226586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:23.229431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 175/355] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.231216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.320828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 181/356] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:23.321931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.357152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.369372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0028d-235d-4c9c-923c-6395e6c97137 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.403962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.417753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.418791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.420373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.420866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.434307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:23.440468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:23.441362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:27:23.442124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:23.446186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 176/357] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:23.447208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.448736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd6d3e-515f-4956-a58c-c06b9703d6dc None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.488897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.489078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.489315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.489557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.490676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.492222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.494234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 182/358] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.496310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.561419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc54b6-303d-4c5d-a6a8-4515622cdbfc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.607487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe4f61-6dc3-4e42-8cb7-b2ee0bf6320f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.623909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.624769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.625392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.626340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.654677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.655503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.656041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.656349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.657167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:23.657620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.659201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.659903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.663140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.673109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 183/359] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.674151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.728904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.731185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1b003-df08-40bf-81a8-7badaecdd086 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.788726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43744607-866a-44a1-9acc-057e01c32221 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:23.834036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:23.834815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.835013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.835290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.837180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=f1260705bec44aa09dcd7d3d097b4913) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:23.837180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:23.838629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:23.845659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 184/360] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=428b94d9bca848c1a1180b457c176272 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:23.846451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.854279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 177/361] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:23.860986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617033-0a56-4e8d-b13d-a0535bbcb234 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:23.861556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617033-0a56-4e8d-b13d-a0535bbcb234 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:23.862002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617033-0a56-4e8d-b13d-a0535bbcb234 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:23.863940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 178/362] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:23.869801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55617033-0a56-4e8d-b13d-a0535bbcb234 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:23.919849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.935635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:23.966457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c2f12-3221-429c-b1e2-dd3fe961f2cd None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.011200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.013004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.013004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.013004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] PATH_INFO: `/v3/users/d85233be71be4ed8a63d102b0108e8f7` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.014021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] RBAC: Authorizing `identity:get_user(user_id=d85233be71be4ed8a63d102b0108e8f7)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.015636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3e117-5642-403b-af42-28f49bb92e0c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.018719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 185/363] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/users/d85233be71be4ed8a63d102b0108e8f7 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:24.020446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.020737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.021009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.038379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55617033-0a56-4e8d-b13d-a0535bbcb234 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.041982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:24.043182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:24.083820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.086123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.086123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.086123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.098716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:24.105289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:24.106150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:24.106953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:24.114657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 179/364] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:24.116079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.140923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 186/365] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:24.142035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.221053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:24.245721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566b056-50a9-49e0-bbf0-8c9cec0b73ce None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.260716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:24.270837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ee49c-b54a-49df-8c71-7177c2b481c4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.289486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.290346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.290849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.291298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.292711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.294448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.296848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 180/366] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:24.298559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.312243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.312866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.313432 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.313707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.314454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] RBAC: Adding query filter params (role.id=a62c9a189f5941a393d9ba582ec6ea49, user.id=d85233be71be4ed8a63d102b0108e8f7, scope.system=all) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:24.315075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.316775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.331772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 187/367] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/role_assignments?user.id=d85233be71be4ed8a63d102b0108e8f7&scope.system=all&role.id=a62c9a189f5941a393d9ba582ec6ea49 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:24.336052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.463239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df460bd-b82a-428e-a72b-c21fc1790c7c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.466022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f15f7-6ad4-4e6f-9df3-c8557ff1e9fe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.483967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:24.532483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.533631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.534105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.534568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.537192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.537806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.538140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.538353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.538785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.539049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:24.539348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.540722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.541084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.543558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 188/368] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:24.549559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.549995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 181/369] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:24.551034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.617906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:24.705241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f61d6-c23e-45ae-abf0-eb7c2caae107 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.756536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.757175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.757175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.757450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.759785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e24fe-6cbf-47fa-b4bf-6b449f67c93d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:24.771573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:24.778524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:24.779672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:24.780849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 182/370] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:24.781510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.817553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:24.818529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:24.819047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:24.819524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:24.820471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=6d626a97866b42ed83934b9ad6319538) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:24.820994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:24.822654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:24.830162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 189/371] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:24.831202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:24.910467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:24.976291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d38ec-a2a6-4afe-ad6c-28481cbca4b8 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.003397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d76fc-aad7-4067-9531-6f32ba8de9d6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.005567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.022148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.023138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.023561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.024075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.025531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:25.026080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.028055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.039298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 190/372] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:25.051380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.090642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.091578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.091870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.092146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.187118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.188925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.210984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204b23-7855-483e-9345-f7f5546a3ae4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.227019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.265102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.265722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.266004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.266281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.353568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 183/373] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:25.363705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.371518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.372599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.373017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.537185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079727f-af20-40fd-ab6a-671d18f90aa6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.540244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 191/374] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:25.546042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.575090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.612693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.613255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.613535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.613809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.615610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.617080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.618730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 184/375] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:25.628860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.760029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bdc7f-804d-4e9a-a3ca-8a11e6ac4ad2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.780970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.835266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e321351-fdb6-4573-bd62-8a0edd458682 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:25.836062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.837028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.837616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.838119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] PATH_INFO: `/v3/projects/86fab351db604d94a6744cf147045d7e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.843519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] RBAC: Authorizing `identity:get_project(project_id=86fab351db604d94a6744cf147045d7e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.845492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.847873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 192/376] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/projects/86fab351db604d94a6744cf147045d7e => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:25.855037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.887993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:25.888967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.889451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.889962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.891953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:25.895879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab1d2e-c858-4546-8306-42637db904d0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:25.898568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 185/377] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:25.900183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb379b-a9d9-47f4-855a-878be455d1af None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:25.900671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb379b-a9d9-47f4-855a-878be455d1af None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:25.901144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb379b-a9d9-47f4-855a-878be455d1af None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:25.903033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 186/378] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:25.904102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb379b-a9d9-47f4-855a-878be455d1af None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:25.966997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:25.993125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e507f-9acf-4c96-809a-803f415a1abb None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.035375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.036448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.036929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.037440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] PATH_INFO: `/v3/groups/91e71bde273946788c5a1b16e45ea71e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.042478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb379b-a9d9-47f4-855a-878be455d1af None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.047458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] RBAC: Authorizing `identity:get_group(group_id=91e71bde273946788c5a1b16e45ea71e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:26.049267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686d0a7-7b2f-4cb4-a98e-52999f8931c3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:26.056301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 193/379] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/groups/91e71bde273946788c5a1b16e45ea71e => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:26.057963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7f2c-91c3-4199-815f-dca82b88ec1e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.058472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7f2c-91c3-4199-815f-dca82b88ec1e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.059053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7f2c-91c3-4199-815f-dca82b88ec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.083749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4ccf7f2c-91c3-4199-815f-dca82b88ec1e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:26.083954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.084516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.084801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.085080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.085819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=6d626a97866b42ed83934b9ad6319538) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:26.086130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:26.086631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4ccf7f2c-91c3-4199-815f-dca82b88ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:26.087587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:26.097394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 187/380] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:25 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=428b94d9bca848c1a1180b457c176272 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:26.098297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:26.107966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:26.225111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 194/381] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:26.289693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf5723-ba76-4844-8b46-6fc3a17af65f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.299160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8c95c-7061-44d8-bb5b-3a7d5daa1153 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.299754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8c95c-7061-44d8-bb5b-3a7d5daa1153 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.300240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8c95c-7061-44d8-bb5b-3a7d5daa1153 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.303906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 195/382] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:26.310244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.310856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.311332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.334393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:26.335944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:26.340365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.340981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.341264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.341538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.342273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] RBAC: Adding query filter params (group.id=91e71bde273946788c5a1b16e45ea71e, role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=86fab351db604d94a6744cf147045d7e) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:26.342587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:26.344059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:26.352114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 188/383] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:26 2022] GET /identity/v3/role_assignments?group.id=91e71bde273946788c5a1b16e45ea71e&scope.project.id=86fab351db604d94a6744cf147045d7e&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:26.434433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 196/384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:26.618603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:26.692022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:26.737578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78257c72-3e5d-4b11-b0f2-a5958029b091 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.780765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.781412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.782039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.782039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.795724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:26.801614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:26.802465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:26.803683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 189/385] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:26.808556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:26.812209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:26.828522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:26.935366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6818a6-7661-443e-b4be-f5e75e07d08e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.943736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d15ae-1c92-4be3-9c14-e5399b3ceaf5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:26.974088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:26.982256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.983035 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.983322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.983599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.988926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:26.989569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:26.989818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:26.990089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:26.994823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:26.994936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:26.995160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:26.996555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:27.000472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:27.001431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:27.002722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 190/386] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:27.010006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.012924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 197/387] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:27.027752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.131658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefdc26-fb7e-4186-987d-feeb289df6af None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.171958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.172643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.172926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.173200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.173878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:27.174180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.176191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.185965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 191/388] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:27.192433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.211837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.265647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.302373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c58e-196e-4845-9e84-3d48479903ce None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.329679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dbd21-9ac9-4744-8e1d-e73f86c7a53d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.346479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.347156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.347438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.349246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.350409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.351933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da140232-47c1-4ad0-b952-c6e0804d4c17 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.353867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 198/389] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:27.355069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524d56-4b63-45e9-9350-809757580ed8 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.355351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524d56-4b63-45e9-9350-809757580ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.355627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524d56-4b63-45e9-9350-809757580ed8 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.357078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 199/390] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:27.363020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524d56-4b63-45e9-9350-809757580ed8 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.378277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.379392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.379673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.380239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.382037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.384415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943594ff-16a9-4961-ad47-bcb94862c386 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.387188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 192/391] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:27.388456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.388792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.389066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.414892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:27.416108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:27.500080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524d56-4b63-45e9-9350-809757580ed8 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.500726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 193/392] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:27.502051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.558073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.559794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.560796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.562554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.565766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.568344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.574552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 200/393] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:27.578512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.581801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.602327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.694974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e0048-c103-427d-b787-c7c69a3fecc6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.757383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38bdd-3ebc-4dff-8487-6178f35d06d0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.763512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.764410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.764883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.765373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.769393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.771110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.778386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 194/394] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:27.785195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.832325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:27.833377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:27.834046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:27.835173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:27.839618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, group_id=7ae96e3eeac24a3fb36ace49fb870ff6, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:27.841496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:27.861328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 201/395] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 24 07:27:27 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:27.862604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:27.948795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.951210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433487c-0432-466f-8f6e-bdcd3d1f2b4f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:27.971635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:27.994640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cc94b-5c4e-4ccc-bfb7-7d97142d84e4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:28.011565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.012172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.012782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.012782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.020099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1260705bec44aa09dcd7d3d097b4913, group_id=7ae96e3eeac24a3fb36ace49fb870ff6, role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:28.021544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:28.037754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.038892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.039439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.039903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.054722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 195/396] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 24 07:27:27 2022] PUT /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/428b94d9bca848c1a1180b457c176272 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:27:28.058943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:28.065863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:28.066846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:28.067647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:28.073660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 202/397] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:28.080449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:28.222452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0c03-84fb-406c-8fae-1c6b5e6204a2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:28.261706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.262235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.262512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.262807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.263481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:28.263779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:28.265152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:28.277005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 196/398] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:28.282995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:28.403473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ce4d-3fcf-493c-9b11-63f18612bf84 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:28.439327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:28.445845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.446673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.447144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.447905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.449125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:28.450801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef7714-0e36-4fd6-8955-0018136c15c6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:28.452956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 203/399] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:28 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:28.463297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:28.463657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a3be-a4ff-400e-bf57-50e22dea8225 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.464150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a3be-a4ff-400e-bf57-50e22dea8225 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.464434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a3be-a4ff-400e-bf57-50e22dea8225 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.466212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 204/400] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:28.472897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.473204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.473463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.494435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:28.498604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:28.530522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:28.596386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 205/401] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:28.612497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701350-c7d6-4de4-ba5a-4d0383c57246 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:28.658340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.659017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.659303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.659639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.660835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:28.662277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:28.668670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 197/402] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:28 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:28.669946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:28.812421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c30f5b-b545-42cd-870d-66f43223ff44 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:28.836082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:28.851308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:28.851969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:28.852242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:28.852509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:28.854405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, group_id=7ae96e3eeac24a3fb36ace49fb870ff6, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:28.855874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:28.883901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 206/403] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 24 07:27:28 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:28.912451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.032968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d700e2-e689-49f7-bb08-cccdc4276dc7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.036039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:29.071493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.072498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.072990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.073473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.084520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d114a61a-14b7-44ff-932a-6bb9f7d23e87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:29.089795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:29.090741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:29.091643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:29.094232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 198/404] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:29.100806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.181407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e299-ed1c-40b4-93c8-edf439cd1d76 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.182046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e299-ed1c-40b4-93c8-edf439cd1d76 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.182533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e299-ed1c-40b4-93c8-edf439cd1d76 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.184397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 199/405] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:29.196210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.196484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.196766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.217124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba125d-c66b-4b3b-9d91-111b4c0b9a81 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.236305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:29.237485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:29.259103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.260183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.260682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.261172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.262054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:29.262687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:29.264293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:29.276234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 207/406] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:29.282600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.364552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:29.368353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 200/407] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:29.432941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657122-f160-455e-bfab-1fa0280f888e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.472624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.473266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.473546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.473819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.476018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:29.477484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:29.479518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 208/408] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:29 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:29.485834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.611092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:29.621580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e310f2-1445-4fd1-aa8a-d650b0746d0c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.667067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.668593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.669370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.669659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.670995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:29.672606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:29.674525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 201/409] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:29 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:29.680905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.751154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:29.863708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a8fed-aa78-406a-8f67-517c29d08605 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.874536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:29.900328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b7a4-fe6a-4a35-9560-99131602f4a3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:29.904379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.905043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.905317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.905646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.912610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a97866b42ed83934b9ad6319538, group_id=7ae96e3eeac24a3fb36ace49fb870ff6, role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:29.914306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:29.933524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 209/410] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 24 07:27:29 2022] PUT /identity/v3/projects/6d626a97866b42ed83934b9ad6319538/groups/7ae96e3eeac24a3fb36ace49fb870ff6/roles/428b94d9bca848c1a1180b457c176272 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:29.958919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:29.960330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:29.960830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:29.961316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:29.976680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:29.982125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:29.983081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:29.984128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:29.987142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 202/411] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:29.995424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:30.123692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08206d7c-1869-4b9d-9c9b-402eed598481 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:30.166992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:30.167857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.168311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.168734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.169570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:30.170034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:30.171665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:30.184710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 210/412] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:30.193266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:30.221071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:30.348890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e830ea0-5961-4e28-bb81-dabcfb6bf2e0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:30.416393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:30.417087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.417312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.417585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] PATH_INFO: `/v3/groups/91e71bde273946788c5a1b16e45ea71e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.418687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] RBAC: Authorizing `identity:get_group(group_id=91e71bde273946788c5a1b16e45ea71e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:30.420147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:30.422002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 203/413] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:30 2022] GET /identity/v3/groups/91e71bde273946788c5a1b16e45ea71e => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:30.427469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:30.461744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:30.546502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd128-d61d-4dab-8c35-0b340dca9f8a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:30.601744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:30.602388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.602664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.602958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] PATH_INFO: `/v3/projects/86fab351db604d94a6744cf147045d7e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.604108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] RBAC: Authorizing `identity:get_project(project_id=86fab351db604d94a6744cf147045d7e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:30.605563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be5174-76ee-4c2a-accf-d8710a20bd7e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:30.607228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 211/414] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:30 2022] GET /identity/v3/projects/86fab351db604d94a6744cf147045d7e => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:30.613431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:30.627585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad403-98ee-49e1-8cc2-e440afb6abdb None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.627857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad403-98ee-49e1-8cc2-e440afb6abdb None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.628122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad403-98ee-49e1-8cc2-e440afb6abdb None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.629770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 212/415] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:30.636253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f13f9-3a42-4fa8-b04c-47f096f848f6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.636532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f13f9-3a42-4fa8-b04c-47f096f848f6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.636807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f13f9-3a42-4fa8-b04c-47f096f848f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.679063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7f9f13f9-3a42-4fa8-b04c-47f096f848f6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:30.680498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7f9f13f9-3a42-4fa8-b04c-47f096f848f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:30.773163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:30.791812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f12-c142-4382-9804-0f951b94d864 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:30.822369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 213/416] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:30.823864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006fa70-b283-4e35-935b-33b46be2d031 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.824258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006fa70-b283-4e35-935b-33b46be2d031 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.824667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006fa70-b283-4e35-935b-33b46be2d031 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.826021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 214/417] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:30.831864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.832257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.832643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.833206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:30.834032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:30.834468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:30.834925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] PATH_INFO: `/v3/projects/86fab351db604d94a6744cf147045d7e/groups/91e71bde273946788c5a1b16e45ea71e/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:30.837245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] RBAC: Authorizing `identity:create_grant(project_id=86fab351db604d94a6744cf147045d7e, group_id=91e71bde273946788c5a1b16e45ea71e, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:30.838962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:30.851787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:30.853179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:30.855278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 204/418] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 24 07:27:30 2022] PUT /identity/v3/projects/86fab351db604d94a6744cf147045d7e/groups/91e71bde273946788c5a1b16e45ea71e/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:30.881424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:30.928861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 215/419] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:31.204793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.209716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.333885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b57b-b0bf-4d08-ad92-fb1e6b05667a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.338774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e2b77-e35b-4bfa-8635-827526e7ac61 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.368946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.369580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.369857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.370123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.376137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.377058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.377556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.378042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.382350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:31.388373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:31.389251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:31.390032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 205/420] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:31.397705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.402229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:31.412781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:31.413698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:31.414499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:31.419901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 216/421] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:31.421959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:31.427424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.451081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:31.533034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9cfc7-be4a-4a82-938f-a4f57f88701f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.549729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c9a2-4e3d-4799-89e1-c13317ac4349 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.576044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.577154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.581471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.581471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.581471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:31.581471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:31.582206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9eca-f578-4c24-b30a-ae44952698ab None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:31.594301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.594790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 206/422] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:31.595451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.595930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.596139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2f4ad-5896-44f6-8350-30e38dd24845 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.596428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2f4ad-5896-44f6-8350-30e38dd24845 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.596752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2f4ad-5896-44f6-8350-30e38dd24845 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.597076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.597997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:31.598497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:31.600199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22734256-11b9-46c9-99f0-651b1080e810 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:31.602513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 207/423] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:31.603519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2f4ad-5896-44f6-8350-30e38dd24845 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.611506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 217/424] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:31.616475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.616761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.617037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.635654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:31.636842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:31.708000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:31.713802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 218/425] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:31.716620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:31.736787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2f4ad-5896-44f6-8350-30e38dd24845 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.754968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:31.777046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.777958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.778420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.778946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.854630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:31.856645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:31.914798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453fe5f-39de-4ce7-bbd4-32d87663810f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:31.931394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:31.950989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:31.951604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:31.951889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:31.952163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:31.968051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 208/426] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:31.974659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.023632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.025078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.038903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.082179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346711d-dadf-4aa7-992d-de91466da893 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.117824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.118826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.119324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.120093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.121241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.122684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.124462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 209/427] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:32.125103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.156444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 219/428] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:32.158285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.180369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.251262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b12d3-1b36-47c2-a219-635dd52f8963 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.287965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.288599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.288706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49230aee-63e6-4034-a3e4-fcc5be68e7fa None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.289115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.289115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.303493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.306434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:32.317040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:32.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:27:32.319036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:32.321213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 210/429] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:32.322320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.329297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.329941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.330222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.330492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.331777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.333230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.338733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 220/430] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:32.338733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.450497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.453446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d873-aefb-4f1e-8aa1-42ac99455873 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.456637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be62f5d-b15f-4875-93ae-cfb11aef8024 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.490552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.491209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.491490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.491605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.491800 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] PATH_INFO: `/v3/projects/f1260705bec44aa09dcd7d3d097b4913` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.492114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.492391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.492662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.492936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] RBAC: Authorizing `identity:get_project(project_id=f1260705bec44aa09dcd7d3d097b4913)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.493317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:32.493614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.494393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.494988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1fcad-6e71-4219-a696-a51b8d77c2ac None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.501417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 211/431] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/projects/f1260705bec44aa09dcd7d3d097b4913 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:32.501417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 221/432] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608db9-bafe-41b2-aed0-9db284017814 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608db9-bafe-41b2-aed0-9db284017814 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608db9-bafe-41b2-aed0-9db284017814 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 222/433] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:32.513710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608db9-bafe-41b2-aed0-9db284017814 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.558794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.620729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe6beb-091c-4441-ae52-5b146b8f0743 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.621608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608db9-bafe-41b2-aed0-9db284017814 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.658109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.658846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.659066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.659297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.659458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.659563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.659720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.659982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.660700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=f1260705bec44aa09dcd7d3d097b4913) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:32.660799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.661003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.662249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.662403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d3814-a77d-4932-8fbd-58b1e1970e3b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.668937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 223/434] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:32.668937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.669136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 212/435] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:32.672055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.673133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.673640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.694990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:32.696587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:32.732448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.786954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 213/436] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:32.788430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:32.820885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2015ae-79ba-46a0-9360-ec4703d159cc None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.835685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:32.867857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.868533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.868833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.869372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.924357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd394252-0708-4658-9a4f-56bf1542b951 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:32.961216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.963173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.970444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:32.971434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:32.971921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:32.972488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:32.974032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=f1260705bec44aa09dcd7d3d097b4913) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:32.974644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:32.976407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:32.983355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 214/437] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=f1260705bec44aa09dcd7d3d097b4913&role.id=428b94d9bca848c1a1180b457c176272 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:33.084611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 224/438] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:33.090049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.127593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.155975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.173232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.223753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85c531-2728-423c-ac94-eb026df0dd2e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.271079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.271789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.272061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.272336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.274351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:33.275918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e6bf5-ce88-4b48-bcfe-b301be079a9b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:33.277855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 215/439] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:33.279118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec978e-4d7f-40b4-b4e6-971f3dd79bf6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.279405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec978e-4d7f-40b4-b4e6-971f3dd79bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.279678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec978e-4d7f-40b4-b4e6-971f3dd79bf6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.281149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 216/440] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:33.282892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec978e-4d7f-40b4-b4e6-971f3dd79bf6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.303683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355db3f-58f0-452b-9e5e-b143e1a8ecb3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.349264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.350060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.350925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.350925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.366931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-aece8447-843f-479e-a871-397d645c7bf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:33.372521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:33.373411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:33.374210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 225/441] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:33.375222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.375531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.376450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.394165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.395708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:33.397376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:33.403262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec978e-4d7f-40b4-b4e6-971f3dd79bf6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.442723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.444408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.444829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.445263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.446404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:33.448024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:33.450881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 217/442] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:33.450881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.479129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.488560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 226/443] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:33.489655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.569306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32092de-d0d1-4c59-af99-a7400cd4beea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.605068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbbdad-5026-40d2-9b7f-5d8d8746a505 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.608333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.608941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.609221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.609494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.610185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:33.610485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:33.612167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:33.625173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 218/444] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:33.627235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.642356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.642992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.643510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.643510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.644268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=c67e3b126ba2441aa233b7fc97e7fee1, scope.project.id=6d626a97866b42ed83934b9ad6319538) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:33.644529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:33.645936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:33.652178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 227/445] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=c67e3b126ba2441aa233b7fc97e7fee1 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:33.654054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.738909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4f47-7e99-4971-9614-f5ad57ebf93c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.769457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.778080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.778360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.778635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.846252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.869454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:33.870948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:33.895716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:33.899207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6723-9a9e-43f7-96fe-b35985b3351e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:33.946863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:33.947507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:33.947780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:33.948044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:33.964674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:33.970550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:33.971482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:33.972886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 228/446] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:33.977901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:33.999612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 219/447] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:27:34.005392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.081527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef70b2-4fe1-4738-8ce5-0e12bfda8bdb None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.101516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43804950-4245-4dd3-86d7-05657aed637a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.104297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.116757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.117358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.117636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.117917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.118573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:34.118922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.120299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.124166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.129100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 229/448] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:34.134685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.135201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.135950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.136233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.136510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] PATH_INFO: `/v3/projects/6d626a97866b42ed83934b9ad6319538` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.137625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] RBAC: Authorizing `identity:get_project(project_id=6d626a97866b42ed83934b9ad6319538)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.139094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.140693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 220/449] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/projects/6d626a97866b42ed83934b9ad6319538 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:34.146387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.237132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5ad04-e007-4262-a5a5-3e952b2f0a9f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.245232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076de6-be6b-4871-9ab0-9f5780b17948 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.270137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.270762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.271040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.271311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.278230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.278876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.279156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.279431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] PATH_INFO: `/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.280535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] RBAC: Authorizing `identity:get_group(group_id=7ae96e3eeac24a3fb36ace49fb870ff6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.282018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.283604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 221/450] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/groups/7ae96e3eeac24a3fb36ace49fb870ff6 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:34.289277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.324918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.338854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.340299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.349054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.390853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0c28a-bbe1-42a7-b5fc-3dd52174be66 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.424283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.424904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.425177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.425444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.426157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] RBAC: Adding query filter params (group.id=7ae96e3eeac24a3fb36ace49fb870ff6, role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=6d626a97866b42ed83934b9ad6319538) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:34.426461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.428006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.433726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 222/451] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/role_assignments?group.id=7ae96e3eeac24a3fb36ace49fb870ff6&scope.project.id=6d626a97866b42ed83934b9ad6319538&role.id=428b94d9bca848c1a1180b457c176272 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:34.440295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 230/452] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:34.446132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.552156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771759-5518-44f5-8836-6fe4cbd66710 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.576555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.593829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.594494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.594794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.595070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] PATH_INFO: `/v3/projects/86fab351db604d94a6744cf147045d7e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.596187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] RBAC: Authorizing `identity:get_project(project_id=86fab351db604d94a6744cf147045d7e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.597614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.599250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 223/453] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/projects/86fab351db604d94a6744cf147045d7e => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:34.604787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.692588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:34.708466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289483b4-c2de-4d14-bd5f-95aeb3ad7fd2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.744017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.744646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.744922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.745195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] PATH_INFO: `/v3/groups/91e71bde273946788c5a1b16e45ea71e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.746317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] RBAC: Authorizing `identity:get_group(group_id=91e71bde273946788c5a1b16e45ea71e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.747785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fefdf0-4fa1-41e7-b072-5269c5352479 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.749347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 231/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/groups/91e71bde273946788c5a1b16e45ea71e => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:34.755822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:34.853026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa156b-d53f-46ca-8c8e-b31984aed936 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:34.885852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:34.886449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:34.886748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:34.887027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:34.887742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] RBAC: Adding query filter params (group.id=91e71bde273946788c5a1b16e45ea71e, role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=86fab351db604d94a6744cf147045d7e) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:34.888047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:34.889488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bb85-e97d-4ef8-8f13-d442597b0668 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:34.895614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 224/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 24 07:27:34 2022] GET /identity/v3/role_assignments?group.id=91e71bde273946788c5a1b16e45ea71e&scope.project.id=86fab351db604d94a6744cf147045d7e&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:37.312393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1269-f764-4515-9693-245c0cb64e55 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.313939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1269-f764-4515-9693-245c0cb64e55 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.314423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1269-f764-4515-9693-245c0cb64e55 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.316292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 232/456] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:37.321270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7546f-0667-47ab-af39-2d2d63291641 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.322068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7546f-0667-47ab-af39-2d2d63291641 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.322545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7546f-0667-47ab-af39-2d2d63291641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.347695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e2e7546f-0667-47ab-af39-2d2d63291641 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:37.349257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e2e7546f-0667-47ab-af39-2d2d63291641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:37.413406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:37.436768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 225/457] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:37.537910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd67aed-416f-444f-8deb-b1c9f610f96b None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.538203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd67aed-416f-444f-8deb-b1c9f610f96b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.538454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd67aed-416f-444f-8deb-b1c9f610f96b None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.539964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 233/458] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:37.552839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604497ee-57cd-4435-94d6-b61e8d1b959f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.553118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604497ee-57cd-4435-94d6-b61e8d1b959f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.553382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604497ee-57cd-4435-94d6-b61e8d1b959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.580115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-604497ee-57cd-4435-94d6-b61e8d1b959f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:37.581232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-604497ee-57cd-4435-94d6-b61e8d1b959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:37.677287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 226/459] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:37.723627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995357df-06cc-4c51-b03e-66fe99d2b214 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.723934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995357df-06cc-4c51-b03e-66fe99d2b214 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.724184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995357df-06cc-4c51-b03e-66fe99d2b214 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.725686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 234/460] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:37.730838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:37.731126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:37.731402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:37.755663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:37.756855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:37.803621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995357df-06cc-4c51-b03e-66fe99d2b214 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:37.844219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 227/461] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:37.956161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995357df-06cc-4c51-b03e-66fe99d2b214 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.001636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.002279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.002553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.002849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.020072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.021597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:38.042405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:38.043352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:38.044806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:38.044806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.044806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:38.044806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 235/462] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:38.049340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.085457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.100533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.117097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.194952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1432c-71e6-4534-8eb2-2ef338ce4b23 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.237502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.238298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.238911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.239189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.240101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:38.240407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.241602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc95d3f-3ec4-4373-8c40-f7622ad7c9a8 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.241796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.257102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 228/463] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:38.258003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.288554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.289561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.290386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.291096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.306065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.308303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:38.330510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:38.333168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:38.335591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:38.335591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.335591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:38.335591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 236/464] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:38.335591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.380324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298853-11ba-43a5-8579-eac89d5ef81e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.387448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.408163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.419465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.420107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.420389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.420660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.431196 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.433145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3cba4afa-6682-4f8e-883c-9c53142961d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:38.448036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:38.448971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:38.449814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:38.449814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:38.449814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:38.449814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 229/465] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:38.450265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42139-6025-4466-96fa-81f3e5af6ee3 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.450544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42139-6025-4466-96fa-81f3e5af6ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.450859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42139-6025-4466-96fa-81f3e5af6ee3 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.452203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 230/466] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:38.452755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42139-6025-4466-96fa-81f3e5af6ee3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.457342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1846d69-e304-4755-9afb-4d36021e6a91 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.493819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.494852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.495134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.495410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.496050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.497489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5082de-e869-4ffe-9527-d00495922863 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.517939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 237/467] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:38 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:38.519183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ec65-ae0a-4013-a0e6-13f2ea76591f None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.519468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ec65-ae0a-4013-a0e6-13f2ea76591f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.519741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ec65-ae0a-4013-a0e6-13f2ea76591f None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.521192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 238/468] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:38.521674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2ec65-ae0a-4013-a0e6-13f2ea76591f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.567145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42139-6025-4466-96fa-81f3e5af6ee3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.613083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.614038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.614327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.614607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.615302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:38.615614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.617026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80b98-9413-4973-a85c-ed73b2b1739d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.634816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 231/469] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:38.634816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.634816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.634816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.649453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2ec65-ae0a-4013-a0e6-13f2ea76591f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.651584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:38.652820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:38.667414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.668472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:38.697188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.698405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.698405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.698405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.701924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:38.701924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.701924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e4918-d3ac-4ab4-8cde-7a5400267f1b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.723070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 239/470] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:38.724229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.724520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.724797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.743900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:38.745102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:38.748858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 232/471] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:38.749942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.819646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 240/472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:38.820910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:38.859032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4ceb-0649-4241-b400-fb226485dab2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.895738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:38.896414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:38.896798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:38.897174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:38.897935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:38.899767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:38.924583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 233/473] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:38 2022] POST /identity/v3/users => generated 248 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:38.982536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a568ba-abcb-493b-87a0-53fd86dcefb9 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:38.985529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.037679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.038319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.038602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.038902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.039619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.041112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.090796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 241/474] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:38 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:39.112055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.116217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.126555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.254825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a65fd-1e95-4ce0-8216-0a44bcae52e0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.274717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fd21-201d-451c-aa54-ce1862186baf None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.300003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.300626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.300948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.301334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.302133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.304053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.314682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.sql.core [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:27:39.314682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:27:39.314682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [parameters: {'id': 'be03bc02cdee43cdbcd20fdff1d9bc37', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:27:39.314682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:27:39.318082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.318331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.318548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.318869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.333246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:27:39.333838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:27:39.334661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application yield resource Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise self.value Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application session.commit() Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:27:39.335643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:27:39.336489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:27:39.337365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:27:39.338279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [parameters: {'id': 'be03bc02cdee43cdbcd20fdff1d9bc37', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:39.340622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:27:39.342591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.346378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 234/475] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 24 07:27:39 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:39.346378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.346650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:39.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:39.364290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:39.367083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:39.367083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:39.367083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:39.367083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 242/476] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:39.369417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.396984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.430957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.472823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a6fa-9e67-4c03-8e8e-90b6016beee4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.480194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98075ce-a205-466d-87c9-156b7651e523 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.510374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.511456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.511903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.512347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.516991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.517681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.518094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.518439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.519240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:39.519571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.521228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.525535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:39.530887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:39.531824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:27:39.532606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:39.535173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 235/478] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:27:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:39.538772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 243/478] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:39.540263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.541313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:39.687075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.721233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a7e5-f897-4608-b9e0-73891ef6a760 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.723332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0854-f0de-41f6-8e1c-fee4d240f501 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:39.778618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.779238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.779524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.779797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.780446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.782480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd19cb-27cd-48e6-acf7-5669dc1d8a01 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.788331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:39.800459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:39.801133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:39.801414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:39.801692 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:39.802372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:39.802680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:39.804821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ff8f7-0334-4a2a-ac57-a8a585c4f8d0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:39.817942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 236/479] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:39.827032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 244/480] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:39 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:40.867581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11475f57-c725-4198-affd-e3014fa7f86b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:40.868114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11475f57-c725-4198-affd-e3014fa7f86b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:40.868509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11475f57-c725-4198-affd-e3014fa7f86b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:40.870219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 237/481] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:40.876727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56d24e-d05d-4c90-b2d1-e483346af9df None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:40.877173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56d24e-d05d-4c90-b2d1-e483346af9df None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:40.877476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56d24e-d05d-4c90-b2d1-e483346af9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:40.907859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6a56d24e-d05d-4c90-b2d1-e483346af9df None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:40.909075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6a56d24e-d05d-4c90-b2d1-e483346af9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:40.992350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 245/482] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:41.398126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11475f57-c725-4198-affd-e3014fa7f86b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:41.539571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11475f57-c725-4198-affd-e3014fa7f86b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:41.590676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:41.591367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.591655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.591933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.592505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdcb8-0dc7-4c75-89f3-72a0850a2411 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.592799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdcb8-0dc7-4c75-89f3-72a0850a2411 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.593077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdcb8-0dc7-4c75-89f3-72a0850a2411 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.594723 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 246/483] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:41.599955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.600427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.600877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.607544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:41.613080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:41.614149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:41.615176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:41.617336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 238/484] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:41.623491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:41.642093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:41.643388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:41.702419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:41.715074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:41.718281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 247/485] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:41.786964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896cc59-a808-4454-a225-dfa522a6ad5a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:41.821600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:41.822221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.822479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.822771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.823436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:41.823732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:41.825108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd539fde-c245-4eb0-9263-db2081c5e74b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:41.833889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 239/486] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:41.839014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:41.883368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c4e6-0f04-4f40-a462-7db4c41bee3c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.883930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c4e6-0f04-4f40-a462-7db4c41bee3c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.883930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c4e6-0f04-4f40-a462-7db4c41bee3c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.885582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 240/487] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:27:41.890274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:41.890549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:41.890852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:41.917650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:41.918930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:41.975465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f5ca8-55d4-4dc0-a95a-0f55f9721775 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.013299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 241/488] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 07:27:42.021779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.022655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.023153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.023622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.033004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.038196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.039995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:42.072831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.073791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:42.074621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:42.074621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.074621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:42.080866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 248/489] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:42.087632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.120589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.136356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.202715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cacd83-fc72-434e-81bb-85eca5528a8a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.212902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a54f0-8dca-45ef-b485-8c0c15229247 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.248556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.249508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.249988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.250488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.251523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:42.252048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.253800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.267349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 249/490] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:42.267490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.269182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.269182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.269182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.274098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.280525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:42.286134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.287024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:42.287822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:42.290162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 242/491] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:42.291150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.393437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cffa9-1d80-4a14-8ee8-1af0c074f573 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.396405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.397304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.401325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4e070-0302-42c6-a8f3-c91a645eaa1e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.429147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.429778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.430060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.430335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.441109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:42.441402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.442236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.442724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:42.445817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.446709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:42.447520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:42.452089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 243/492] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:42.452949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.455704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.457360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:42.471972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.472908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:42.473827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:42.476459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 250/493] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:42.477562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.560519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bff3d-53e1-4e04-bb06-28e77420803e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.586961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17991-05e1-4d38-b5c8-e9148f892c8e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.597200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.597805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.598080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.598353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.599044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:42.599343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.600693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.608399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 244/494] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:42.609174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.624117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.624718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.624993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.625262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.626483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:42.626818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.628186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c50d-7b1b-428d-ada6-16bdb17eb425 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.633515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.636331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 251/495] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:42.637621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96953c-7f1c-4af8-ab28-671b48c6eee1 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.637902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96953c-7f1c-4af8-ab28-671b48c6eee1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.638175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96953c-7f1c-4af8-ab28-671b48c6eee1 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.639519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 252/496] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:42.640484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40e5de-1a44-4067-9611-e855ac25aeb5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.640792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40e5de-1a44-4067-9611-e855ac25aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.641242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40e5de-1a44-4067-9611-e855ac25aeb5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.642952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 253/497] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:42.643694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40e5de-1a44-4067-9611-e855ac25aeb5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.668601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.712993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34807-94ae-4fdd-ba74-524d8d85cbf9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.745617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.746024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40e5de-1a44-4067-9611-e855ac25aeb5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.746217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.746499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.746802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.747524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:42.747824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.749253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.779459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.780062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.780342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.780612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.785918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 245/498] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:42.786758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:42.791345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.792897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-edcaceb3-4e62-452e-930c-5fbda36c0303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:42.808241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.809118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:42.810048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:42.810048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.810048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:42.810048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 254/499] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:42.810687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953376b-cfd9-416b-8476-4017e06479d1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.810935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953376b-cfd9-416b-8476-4017e06479d1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.811215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953376b-cfd9-416b-8476-4017e06479d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.829515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5953376b-cfd9-416b-8476-4017e06479d1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:42.830754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5953376b-cfd9-416b-8476-4017e06479d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:42.872088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.891803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9259-5279-4634-b3b7-3b06f5019757 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:42.896146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 255/500] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:42.897463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682d9c-e873-4836-952d-d8f23522562c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.897744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682d9c-e873-4836-952d-d8f23522562c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.898013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682d9c-e873-4836-952d-d8f23522562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.908510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:42.920492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-48682d9c-e873-4836-952d-d8f23522562c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:42.921670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-48682d9c-e873-4836-952d-d8f23522562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:42.934659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:42.935316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:42.935766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:42.936045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:42.947641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:42.949096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:42.964516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:42.965416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:42.966956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:42.966956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:42.966956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:42.966956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 246/501] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:42.966956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.003577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 256/502] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:43.003577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682d9c-e873-4836-952d-d8f23522562c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.103472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682d9c-e873-4836-952d-d8f23522562c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.103654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7fe65-d8cc-4f7d-a490-789d36e7f95c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.140777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.141914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.142775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.143478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.146478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.147944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.149509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.149944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.150307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.151054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:43.151438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.152952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.154493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.169429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 257/503] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:43.175109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.228533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.230214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.274599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46609580-04bf-4f80-a487-d342eac9b7b3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.309004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.309608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.309890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.310170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.310855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:43.311160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.312526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.324590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 258/504] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:43.325377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.333899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 247/505] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:43.334751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.371226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.386188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.431390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2448-2009-41e0-b5a3-e706cf1af6e1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.436398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36435c7c-16ef-45b2-b604-4461fc8e63aa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.465346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.465963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.466235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.466498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.468545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.469694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.469981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.470249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.477125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.477366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.478562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.478814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:43.491407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:43.492313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:43.493180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 248/506] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:43.493723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:43.494336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:43.495380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:43.496205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:43.496205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:43.496205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:43.496205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 259/507] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:43.496205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.590824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.596260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab380d-72d9-4a78-a1de-ded213e1f410 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.598135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411d52-8a8f-4a55-91a5-d25445db1446 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.603271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.630050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.630208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.630606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.630852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.630955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.631130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.631215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.631400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.638538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.639991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.641641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:43.646735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:43.647653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:43.648427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 249/508] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:43.649402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:43.651425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:43.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 260/509] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:43.654909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.748261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b12c6-33ec-44d1-9eab-0a2288eeb10e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.753192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8f56-9ab7-4c06-aa95-5f6163ce1bab None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.780846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.781453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.782286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.782564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.786990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.787577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.787850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.788116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.788833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:43.789132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.790599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.792841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.794275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:43.808126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:43.809466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:43.810351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:43.810351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:43.810351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:43.810351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 250/510] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:43.810351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.822404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.827506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 261/511] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:43.828404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.854590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:43.916289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307f58a-d685-4b51-92e0-652302a4c247 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.928666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffb95-5a95-4892-8efe-b839d09ce1c4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:43.948446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.949033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.949311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.949578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.950209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:43.950506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.951905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.962640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:43.963273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:43.963554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:43.963726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 251/512] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:43.964123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:43.964597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:43.964799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:43.965106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:43.966670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:43.976376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 262/513] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:43.977166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.029674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.061236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.064544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98a42-7f1d-4f8b-adcc-afe4750e1f90 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.076156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d94f3-3776-4a26-89b0-1a0f48a4ad59 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.096984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.097577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.097854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.098132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.098886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:44.099187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.100598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.108896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.109497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.109806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.110082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.110746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:44.111051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.112415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.116953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 252/514] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:44.117743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.127206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 263/515] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:44.127991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.218308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b1869-5c78-494b-b4f2-7d6b2bf707c5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.226475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeede9b-31ff-4925-800e-98f9dc308497 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.240531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.251832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.252423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.252702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.252968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.260662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.261327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.261631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.261903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.262527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.263991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.284976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 264/516] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:27:44 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:27:44.285846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.301574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.327101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.329267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.410531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4324f-aa65-4ca2-b5b8-16c4b9e4d790 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.441754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 253/517] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:44.442956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.456569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.458799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.458799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.458981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.472141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.473726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.474019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:44.490611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:44.491745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:44.492763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:44.492763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:44.492763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:44.492763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 265/518] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:44.494044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.554386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.554576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4d1c-8d40-4bc6-ad15-b9a11c10f691 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.590974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.591490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.591766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.592036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.614819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b5c1f-3890-4fac-9d59-a2b718deba35 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.651353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.651983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.652261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.652535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.660116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.661545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.666219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.667718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:44.673804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:44.674638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:44.675796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 266/519] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:44.676256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.697986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.784778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 254/520] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:44.784943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab2d6-555c-4873-88a0-e7948e7db2b0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.786046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.803334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.829708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.830470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.830786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.831066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.844910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.846918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:44.862087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:44.863068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:44.863896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:44.863896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:44.863896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:44.867906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 267/521] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:44.868484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.893032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a231885-ded8-4329-a2e2-08c5e8b186f6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:44.918203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:44.930976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:44.931580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:44.931918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:44.932198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:44.932849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:44.933154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:44.934581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:44.951048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 255/522] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:44.951876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:44.979644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f5888-a05b-4e59-8f7b-672d51b3a2d6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.015290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.015888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.016187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.016456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.017139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.017437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.019103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.037856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 268/523] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.038710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.059470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57c6c-53a3-4ed9-976d-7ff66410534c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.066965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.099017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.099647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.099931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.100236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.111754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.113782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:45.128499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:45.129393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:45.130350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:45.130350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:45.130350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:45.130350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 256/524] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:45.130350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.152588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8ac3-f618-4ecc-8a59-73100f03370d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.168914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.191729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.192753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.193256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.193532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.194175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.194475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.195927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.208665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 269/525] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.209423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.242639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08908d7a-9d66-4219-a9ee-adcb485a0ef0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.286231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.286890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.287710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.288021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.298007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.298261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.299731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.314064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea9c5-32da-4e6f-a69a-9d01ce4d966d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:45.314973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:45.315867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:45.316829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:45.317891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 257/526] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:45.318090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.353690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.354550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.354857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.355132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.355854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.356163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.357618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.363974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 270/527] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.367743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.437332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.445777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267dcbf-2955-4a3e-a553-46f905e60a33 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.496332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.497162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.497711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.498089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.499330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.499752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.502088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.521320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 258/528] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.522491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.529614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99959c48-56eb-4d7a-a431-2a9a1ea23d9f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.565838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.567234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.567658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.568095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.576843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.578572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:45.597941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:45.599122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:45.599946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:45.607190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 271/529] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:45.608622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.626906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.664069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6b441-fd62-4c9d-a9ff-96577d2755d7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.708614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.709226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.709508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.709781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.710517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.710975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.712911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.736721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 259/530] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.738015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.743989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b399a2-e77d-4e96-9129-94f9b6f8c2b9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.756739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.787883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.788633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.788919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.789193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.796320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.798666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:45.810565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:45.811484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:45.812310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:45.817102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 272/531] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:45.818023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.864549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521458-a3c2-4992-8f7d-30d9e32ad73a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.895564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:45.901005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.901887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.902385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.902900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:45.904091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:45.904591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:45.906206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:45.930185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 260/532] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:45.931296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807e06-3cd6-432b-90d8-3e15e3cc4df1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:45.932023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:45.975926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:45.976570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:45.976852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:45.977126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.026555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.064029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74b281-3c27-47ab-b265-e3876a7e8d2c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:46.069208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.070615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.111188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:46.112644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.113156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.114059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.115403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:46.115908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.118030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.138144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 261/533] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:46.139474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.146351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.185095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 273/534] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:46.186485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.264247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b466-9f27-410d-8d88-e3adaff3c43b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:46.304350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeacc1f-8816-49bd-a906-03cacaa1eb11 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:46.304886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:46.305561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.305892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.306245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.307081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=ea288fe3842541999821bd863ca9d0c2) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:46.307463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.308899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.310818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.316241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 262/535] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/role_assignments?user.id=ea288fe3842541999821bd863ca9d0c2&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:46.317048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.360994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:46.361620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.361899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.362171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.363731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=923067fd0c864ca9bb202ee9585062f3) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:46.364060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.365703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac566039-b6c9-4c7e-9008-2e2d7eedc5cd None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.374190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 274/536] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/role_assignments?user.id=923067fd0c864ca9bb202ee9585062f3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:46.376199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54303428-b04a-44d3-858b-3db911e5e0be None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.376484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54303428-b04a-44d3-858b-3db911e5e0be None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.377097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54303428-b04a-44d3-858b-3db911e5e0be None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.382535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 275/537] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:46.388425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.389009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.389437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.430378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:46.431810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:46.481841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.519709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6acd-a5e1-4af0-ab14-de7e0beb9368 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:46.570835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 276/538] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:46.580618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:46.581267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.581547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.581833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.594664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.596225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:46.611903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:46.612964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:46.613990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:46.613990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:46.613990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:46.613990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 263/539] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:46.622001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.781720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19183d1a-e0db-4d74-b4b1-5b4a90cfd58c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:46.822099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:46.822803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:46.823087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:46.823361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:46.824014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:46.824318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:46.825695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:46.837968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 277/540] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:46.843597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.877275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:46.914197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.914940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:46.972366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272d4a-51fb-4476-88bb-63e9fdbc08fe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.010264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.010938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.011227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.011512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.019013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.020757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:47.044413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:47.045289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:47.046105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 264/541] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:47.054575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.079608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9533-dfdb-4e68-b5b6-d3304af23088 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.151591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.152147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.152425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.152707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.178228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:47.186791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:47.187649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:47.188725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 278/542] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:47.194534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.252349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7527bf-84fb-4f12-9870-d49ac210999d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.275470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:47.322103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.323400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.324576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.325082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.325980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:47.327047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.329361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.355516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 265/543] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:47.362837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.384065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868101e-53bb-4e2b-9791-fbe0e8c77924 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.387011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:47.428763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.429307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.429583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.429856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.430527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:47.430856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.432307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.448050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 279/544] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:47.454161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.564523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658719b-a81a-44e1-8afa-62085b720e64 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.587519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203d85b-3f4c-4ccd-9f2e-70ae6a785bab None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.613452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.614166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.614506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.614870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.615801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=ae3cfe87a8b347729eb0cbb4f718d394) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:47.616259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.617773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830e11c-beb4-41b0-8766-06eba65ee125 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.624287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.624435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 266/545] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/role_assignments?user.id=ae3cfe87a8b347729eb0cbb4f718d394&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:47.624887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.625170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.625474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.626122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523a45-9b32-42a6-ade9-dd4768e0c70c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.626467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523a45-9b32-42a6-ade9-dd4768e0c70c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.626890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523a45-9b32-42a6-ade9-dd4768e0c70c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.632758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 267/546] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:47.634357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.634762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.635264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.637364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.639001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.647163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:47.656260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:47.658083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:47.659347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:47.659347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:47.659347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:47.659347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 280/547] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:47.659708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:47.660868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:47.663652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.694554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:47.744821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 268/548] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:27:47.773984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb905eb-3747-489f-a499-c32706cd510f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.810025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.810654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.810965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.811238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:47.811894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:47.812196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:47.814173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:47.828279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 281/549] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:47.834136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:47.954742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876be413-2f56-44cb-87af-d5c64ddb3503 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:47.992874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:47.993639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:47.993837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:47.994118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.001498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:48.002976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:48.014732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:48.015614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:27:48.017225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:48.020567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 269/550] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:48.022010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.104787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.148021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.149133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cfeec-31b5-4357-8b8c-65bd3c580350 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:48.195870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.221699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:48.222637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.223135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.223597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.224547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:48.225203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:48.227060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:48.255177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 282/551] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:48.261223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.342117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae500-0d4a-480e-8fc6-93855e22d0bb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:48.416300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:48.416933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.417221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.417506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.419439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c010-b7e6-47e0-a53e-39d1814e46b3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:48.437507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.439817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:48.448546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:48.449484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:48.450446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:48.451116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 270/552] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:48.456066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.463285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:48.465613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.466091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.466636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.604892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.619801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:48.621459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1a829-aaab-4edd-8135-2cad26471d7f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:48.625942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b437f4-4db2-44c6-951d-ab9ff9c85900 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:48.669289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:48.670222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.670887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.671792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.672950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:48.673427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:48.675612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e474e-1b06-4cca-8c12-10c5263b6687 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 271/553] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378812-1450-4f3e-a224-bef95c83ec4d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378812-1450-4f3e-a224-bef95c83ec4d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378812-1450-4f3e-a224-bef95c83ec4d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 272/554] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:48.689319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57197151-9368-47b1-9bcf-87f19ffeb82e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.689763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57197151-9368-47b1-9bcf-87f19ffeb82e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.690404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57197151-9368-47b1-9bcf-87f19ffeb82e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.691531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 273/555] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:48.695801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57197151-9368-47b1-9bcf-87f19ffeb82e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.735528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 283/556] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:48.736785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fdc0-5496-47fc-8307-1062d0c3c961 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.737068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fdc0-5496-47fc-8307-1062d0c3c961 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.737332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fdc0-5496-47fc-8307-1062d0c3c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.754992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-92f7fdc0-5496-47fc-8307-1062d0c3c961 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:48.756173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-92f7fdc0-5496-47fc-8307-1062d0c3c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:48.767483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.821598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57197151-9368-47b1-9bcf-87f19ffeb82e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:48.829637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 284/557] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:48.832107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.833013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.833443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.858660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:48.859785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:48.860405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:48.860626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:48.861005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:48.861391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:48.873594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:48.875614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:48.883430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:48.892850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:48.893976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:48.895050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:48.895050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:48.895050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:48.901403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 274/558] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:48.902435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:48.962239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 285/559] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:48.962239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.053308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd38d-af95-441d-911e-f9ab12c5b3e5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.098981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.118627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.119224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.119502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.119778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.131666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc2b7-8cec-41ec-8c59-3e13e35f62c7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.132129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.133458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:49.152823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.153843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:49.154752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:49.154752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:49.154752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:49.158817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 275/560] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:49.159750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.169475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.170305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.170757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.171168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.171955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:49.173007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.174540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.186636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 286/561] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:49.187743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.214916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.272363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5857bc-86d8-4e09-b960-97dcdb11acf4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.291593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b5ad5-66d8-48b0-8cd8-a3cbc9eba8b1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.310182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.310837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.311118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.311391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.322006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:49.326304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.326946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:49.327221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.328082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:49.328920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:49.329562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.329562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.329562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:49.329562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.330315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.332887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.333167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 276/562] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:49.334147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.343586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 287/563] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:49.345044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.444259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848c36-672d-4e5f-b7ad-0f71b3eb1982 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.453158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f1d8-13d3-43ef-b2f3-aa61158cf502 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.457610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.483405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.484025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.484299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.484564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.486706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.487345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.487628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.487902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.493823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.495749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.498081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:49.503400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.504355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 288/564] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:49.507515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:49.509075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.510031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:49.511024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 277/565] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:49.511633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.572405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.629218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d748-c50c-45fb-8ce7-49a7ff1f4303 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.631608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfb3bc-b1e2-40f9-a267-8cc90c83e3df None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.664535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.664959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.665570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.665869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.666050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.666202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.666677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.666890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:49.667195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.667492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.668565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.675616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.677265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 289/566] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:49.677497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.678067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:49.693759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.694868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:49.695750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:49.700582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 278/567] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:49.701421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.732403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.788177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cc29d-7c99-43dd-a751-5ef3fa637ce3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.806399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.808908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d5872-a51d-4df1-98b6-4c7eb743421b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.824217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.824820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.825093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.825362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.826019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:49.826319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.827711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.836143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 290/568] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:49.836934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.848767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.849378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.849665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.850244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.851005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:49.851306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.852715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.869280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 279/569] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:49.870067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:49.937427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ded0c-2861-488c-9578-de2f0e43fb6e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.955827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:49.967954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2b63-d232-438f-816d-8b03e26a61bc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:49.972039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:49.972670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:49.972950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:49.973221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:49.983545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:49.985049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-b10e297f-91bd-4c93-90c0-edb1117e187e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:49.998822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:49.999788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:50.001201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:50.001201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:50.001201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:50.001911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.002508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.002818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.003086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.003799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:50.004100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.005104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 291/570] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.005500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.006328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fa63-8ba6-49a8-8a33-b3732f6563d5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.006608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fa63-8ba6-49a8-8a33-b3732f6563d5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.006904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fa63-8ba6-49a8-8a33-b3732f6563d5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.008146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 292/571] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:50.008822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469fa63-8ba6-49a8-8a33-b3732f6563d5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.017903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 280/572] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:50.018916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.040211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.110848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469fa63-8ba6-49a8-8a33-b3732f6563d5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.117528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a804e-dbed-48ff-8f4e-fecda122baa1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.143995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.144620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.144905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.145181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.150114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.150766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.151047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.151315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.155535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.157522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.162482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.163935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:50.171456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:50.173052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:50.174046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:50.174046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:50.174046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:50.174242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.175476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 293/573] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.176384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d2ae56cb-2734-4f70-9f0f-b3c885a4fefa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:50.179308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:50.180186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:50.181227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:50.181227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:50.181227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:50.181227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 281/574] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.181995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.182492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.182801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.201891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:50.203107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:50.264891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.267542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 282/575] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:50.268359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.279851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84be70a-2311-4996-a8a3-052298aa7734 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.317527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.318133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.318411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.318682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.319344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:50.319643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.321031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.333047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 294/576] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:50.333842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.396577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfa4ea-e90f-4df0-80dd-d8b5f8c6267f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.432573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.433610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc907a-e507-4299-a507-a2717c5be92b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.441805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.442679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.443537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.444009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.445237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=5c30642f91c54258b4ceec9005408be3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:50.445978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.447783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.455332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 283/577] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/role_assignments?user.id=5c30642f91c54258b4ceec9005408be3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:50.456518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.474532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.475199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.475477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.475752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.476412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:50.476713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.478268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.491780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 295/578] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:50.492820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.496765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.596888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2f365-bf78-45ad-9c93-b1b5574fd39a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.609777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43155-ed33-46ad-a9fe-b437461a4261 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.637561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.638317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.638603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.638898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.639547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:50.639848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.641733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.649332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.649963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.650275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.650729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.659054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 284/579] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:50.659865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.659994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.661331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:50.673300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:50.674286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 296/580] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.676157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.725643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.742357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.772229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adebedd-d7c1-41e3-a77a-51bbb1e33d35 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.784030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089bd-fddb-4e4d-a6b8-120d12055893 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.807042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.807696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.807975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.808238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.816307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.817829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.820008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.820710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.821091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.821464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:50.832116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:50.833037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 285/581] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.834037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.838438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:50.845534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:50.846453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 297/582] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:50.851301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:50.942838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3e13-a6f0-4a4c-b607-fe84337c12ea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.959581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:50.968751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9a056-ddde-4179-90d0-e98f534aa711 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:50.981333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:50.982049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:50.982403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:50.982784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:50.990466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:50.992014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:50.995812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:51.004995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:51.006044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:51.007142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:51.008377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 286/583] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:51.008913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.015301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.016020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.016379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.016719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.017527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.017893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.019408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.039094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 298/584] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.039722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.119120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09b233-4c5e-4307-952a-cc0cc6d8a889 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.150011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffdc4a-9a1e-4466-b9da-8d49883b56b1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.153893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.154504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.154813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.155088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.155817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.156122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.158154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.175220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 287/585] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.175882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.193064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.193858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.194141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.194524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.195112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.195413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.196766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.209964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 299/586] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.209964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.215527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.257149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.287533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c65f-bc29-480d-b34b-58193fd16be6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.319272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48060e-d35e-4d44-a8e4-db628ede18fc None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.323220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.323958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.324238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.324505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.325183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.325480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.326912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.343265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 288/587] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.343265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.358290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca3530-4801-4662-805a-5293532d848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.359024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3530-4801-4662-805a-5293532d848a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.359371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3530-4801-4662-805a-5293532d848a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.360242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca3530-4801-4662-805a-5293532d848a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.371741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca3530-4801-4662-805a-5293532d848a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.373062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca3530-4801-4662-805a-5293532d848a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-22ca3530-4801-4662-805a-5293532d848a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:51.389672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:51.390653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:51.392109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:51.392109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.392109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:51.392109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 300/588] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:51.392938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca3530-4801-4662-805a-5293532d848a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.449992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.460961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581905-d5a2-4df2-9314-4746d6573736 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.493561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.498206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.498863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.499139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.499408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.501079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca3530-4801-4662-805a-5293532d848a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.511023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.512469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:51.528366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:51.529383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:51.530242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:51.530242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.530242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:51.530242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 289/589] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:51.530242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.544599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.545233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.545513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.545789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.557320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.558804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:51.573347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:51.574295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:51.575706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:51.575706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:51.575706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:51.575706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 301/590] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:51.575706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.644443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2a036-56c1-4018-b072-82ef6f921a1f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.676837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.677487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.677765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.678039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.693368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=cff82f8d12d842d1baab39bd73ff54cd, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.693787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cc6c-c210-40e7-ba19-f0c717aa7c2b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.694850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.697110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.710150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 290/591] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:51 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:51.716697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.737867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.738492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.739163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.739163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.739747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.740103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.741598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.756260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 302/592] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.757187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.771049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:51.861837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a31a4b-9058-4937-b680-e1a8fdc8a6c8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.900918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f3510-b661-47f6-9ab2-69a75f002247 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:51.910320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.911104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.911939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.912326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.913088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.913898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.915773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.936288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 291/593] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.936288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:51.952694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:51.954545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:51.955260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:51.955684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:51.956807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:51.957267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:51.959234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:51.976619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 303/594] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:51.977953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.012643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.070493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.078093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66d39-c32f-4b0d-9f68-51d130298376 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.116848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf48a4-6480-4e1f-ad54-95904c3856c4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.124300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.125059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.125509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.126424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.135921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.138189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:52.154919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:52.155979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:52.157447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:52.162523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 292/595] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:52.163341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.163786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.164667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.165145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.165634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.175411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.177542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:52.193205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:52.194291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:52.195303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:52.200904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 304/596] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:52.200904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.296898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409f1b1-d32f-4c25-9751-27d6954e143f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.314971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.332419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e113-f46b-4f69-9c45-c0ab5b94a34b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.341016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.342164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.342445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.342742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.353676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.355420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.368391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:52.371964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:52.373201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:52.374392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:52.378125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 293/597] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:52.378983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.380234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.380842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.381116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.381387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.382185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:52.382517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.384099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88b47e-bacb-402c-ace5-91fb1e794f97 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 305/598] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac511d-d997-4f9d-9a00-4996203e8407 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac511d-d997-4f9d-9a00-4996203e8407 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac511d-d997-4f9d-9a00-4996203e8407 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 306/599] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:52.405222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac511d-d997-4f9d-9a00-4996203e8407 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.483901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d521-c925-443d-b9ef-01693f855791 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.511753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac511d-d997-4f9d-9a00-4996203e8407 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.519009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851681c-f801-4396-b972-451a216ec29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.519616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851681c-f801-4396-b972-451a216ec29f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.519895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851681c-f801-4396-b972-451a216ec29f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.520171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851681c-f801-4396-b972-451a216ec29f None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.520859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851681c-f801-4396-b972-451a216ec29f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:52.521163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851681c-f801-4396-b972-451a216ec29f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.522584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851681c-f801-4396-b972-451a216ec29f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.539620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 294/600] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:52.539620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851681c-f801-4396-b972-451a216ec29f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.556219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.556820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.557125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.557396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.558116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:52.558414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.559877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2329-9607-4fb2-bcae-dce22b50acc2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.574653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.584414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 307/601] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:52.584414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.584414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.584414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.600121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:52.601406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:52.650145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.652586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851681c-f801-4396-b972-451a216ec29f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.677046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 308/602] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:52.678130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.698140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.698841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.699167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.699441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/ea288fe3842541999821bd863ca9d0c2/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.715445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=ea288fe3842541999821bd863ca9d0c2, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.716983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.735190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 295/603] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:52 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/ea288fe3842541999821bd863ca9d0c2/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:52.735190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.850014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661f98-04bf-45ce-b507-446d47f522ce None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.890467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:52.892061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:52.892577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:52.893636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/923067fd0c864ca9bb202ee9585062f3/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:52.916214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=923067fd0c864ca9bb202ee9585062f3, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:52.918896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:52.936231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:52.945453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 309/604] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:52 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/923067fd0c864ca9bb202ee9585062f3/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:52.957449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:52.959138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bec4-f38b-4e99-acc5-fa7ad9d45c15 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:52.993374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.002507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.003258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.003542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.003816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.016039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.018144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:53.036036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:53.037036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:53.038549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:53.038549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:53.038549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:53.038549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 296/605] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:53.043294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.094477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7a73-a730-4aee-a1eb-f446bf671fe0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.137767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.138821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.139300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.139822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.151598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:53.156390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:53.157880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:53.158775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:53.159416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fe58d-e0f1-4baa-9061-45436271db4f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.162878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 310/606] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:53.164062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.200325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.200961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.201244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.201545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.202254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:53.202550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.203961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.206602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.218378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 297/607] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:53.227701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.295879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.323792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb269d-2330-41ea-a65b-c3380d611aa2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.391982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.392716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.393182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.393455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.394131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:53.394427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.395826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.414959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 311/608] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:53.419103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.430047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed973-d018-4801-8e05-04cee147e7bc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.492721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.493285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.493560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.493828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.509616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.511106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:53.523496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:53.524416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:53.525271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:53.528956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 298/609] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:53.530571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.586827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670602-5e5e-4b03-b09e-7b70f079431a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.624978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.630862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.631368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.631869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.648869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.652071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.670712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.671917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:53.677068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:53.678025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:53.678945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:53.678945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:53.678945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:53.679616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 312/610] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:53.689245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.749349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f598ef9-8cf6-4fc0-ad57-8f115b1d649b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.795766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.796371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.796653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.796919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.797610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:53.797913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.799408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fd695-f886-4f57-a139-11b29fa2f0d7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.813986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 299/611] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:27:53.815208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17576fb2-9ad8-4ae2-bfbf-980027e6d451 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.815594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17576fb2-9ad8-4ae2-bfbf-980027e6d451 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.815873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17576fb2-9ad8-4ae2-bfbf-980027e6d451 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.817456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 300/612] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:53.822155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17576fb2-9ad8-4ae2-bfbf-980027e6d451 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:53.835563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765431e8-aba3-4e30-8494-af4de43ea0a5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.871171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.871789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.872069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.872343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.872983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:53.873279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:53.874932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5f955-eedd-4103-9dc3-5c039caf6099 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:53.888159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 313/613] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:53.889343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.889623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.889897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.908683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:53.909980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:53.934481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17576fb2-9ad8-4ae2-bfbf-980027e6d451 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:53.953053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.971185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:53.975027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:53.975731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:53.976060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:53.976341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/ae3cfe87a8b347729eb0cbb4f718d394/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:53.983080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 314/614] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:53.983921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.005758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=ae3cfe87a8b347729eb0cbb4f718d394, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.007205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.037382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 301/615] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:53 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/ae3cfe87a8b347729eb0cbb4f718d394/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:27:54.092527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eac677-8ec3-4249-85d4-63a9f95eeb24 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.131807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.132459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.132736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.133004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.142032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.143546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:54.156541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:54.159126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:54.160261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 315/616] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:27:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:54.166931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.312963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4b3c-876f-4880-b63a-7e11e3679781 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.353830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.372164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.372776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.373053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.373325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.374054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:54.374359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.376420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.408317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 302/617] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:54.417790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.439102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:54.445006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:54.514168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7366a-b3d3-4b83-ad57-13944dd3afd5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.571243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.571808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.572089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.572366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.585489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a89c-5643-4afb-92cd-5d034952bd8b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.592500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:54.598321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:54.599270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:54.600176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:54.604958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 316/618] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:54.607418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.652044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.652683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.652963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.653236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.669397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.670889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:54.698174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:54.699225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:54.700250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:54.700250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:54.700250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:54.700921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 303/619] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:54.708412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.795096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce035-cd0f-4e0d-9247-381818fec617 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.811057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:54.844635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:54.859036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.860184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.860475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.860749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.861429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:54.861733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.863138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.879637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 317/620] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:54.883408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:54.892685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aa718-e6d3-413e-98ac-4488031ce76b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:54.955425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:54.956056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:54.956340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:54.956618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:54.957326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:54.957634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:54.959477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:54.974446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 304/621] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:54.984392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.068984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e960-d1a2-40af-b82f-dfeca9318ff6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.109522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.110606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.111108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.111646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.122528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.124262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-af709bff-c067-4d16-9393-abb62f4b94b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:55.138876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:55.140187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:55.141589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:55.141589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:55.141589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:55.143654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 318/622] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:55.145321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa23-aede-48be-8fd0-35625528ae75 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.145604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa23-aede-48be-8fd0-35625528ae75 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.145879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa23-aede-48be-8fd0-35625528ae75 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.147396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 319/623] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:55.152265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbaa23-aede-48be-8fd0-35625528ae75 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.180618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae878d-b689-45d4-9390-e22bb042d35a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.181681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.217840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.218474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.218797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.219691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.222486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.227358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.229109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-50dc5547-83dc-47ab-aee2-6974eba1dcdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:55.241600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:55.242747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:55.243718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:55.244389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 305/624] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:27:55.250826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.250826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.250826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.265629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:55.267414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:55.275036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbaa23-aede-48be-8fd0-35625528ae75 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.312304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.312980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.313523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.313791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.314437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:55.314865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.316342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.330450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 320/625] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:55.331424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.348969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 306/626] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:55.349833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.423182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.440747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e3c5a-f3d5-4ed2-98d9-741f13b8ed26 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.459342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550238-1d31-4ca0-bdb1-9420e72986a3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.477266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.477969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.478346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.478623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.479338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:55.479637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.481071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd84526-3cd1-472d-9afe-edcbb16d5655 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.487727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.497257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.497889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.498168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.498441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.500706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 321/627] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:55.502621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919debe5-d236-48b1-a196-6bd4b9316ccc None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.502950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919debe5-d236-48b1-a196-6bd4b9316ccc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.503290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919debe5-d236-48b1-a196-6bd4b9316ccc None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.505092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 322/628] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:55.506013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919debe5-d236-48b1-a196-6bd4b9316ccc None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.508275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.510087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3ec28eb6-288a-43d7-a34c-cf5c52949cee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:55.522345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:55.524601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:55.526867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 307/629] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:55.534748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.534748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.534748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.550583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:55.551894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:55.617857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919debe5-d236-48b1-a196-6bd4b9316ccc None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.635052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 308/630] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:55.636383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.654610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.655355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.655629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.655900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/5c30642f91c54258b4ceec9005408be3/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.674597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=5c30642f91c54258b4ceec9005408be3, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.676191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.678640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.698964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 323/631] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:27:55 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/5c30642f91c54258b4ceec9005408be3/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:27:55.700195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.749331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3584c-975f-4f51-a51a-4c7fc2e91f0f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.768252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:55.793554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.794546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.795060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.795733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.798878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:55.799459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:55.801137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:55.826304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 309/632] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:55.832880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.911773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee7ff2-18d6-4bf1-be70-b2f2d2751812 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.947149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643243-6362-4a07-8920-3db183bbb6ac None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:55.949658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.950727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.950727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.950924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:55.962188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:55.967689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:55.968619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:55.970046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 324/633] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:55.970777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:55.990376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:55.991202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:55.991313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:55.991545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.040071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.054118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.070750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.072309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.116325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bd1c4-1fd2-4167-911a-e593203d9442 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.169053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.170669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.171214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.171934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.185704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:56.187479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 310/634] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:56.189155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:56.193727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:56.194667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:56.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:56.196163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 325/635] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:56.198883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.275400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.307122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c0ff-0f4b-4a3d-bf22-dd69932ee745 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.317492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144b265-00a0-4482-b987-8ce63bcf4ece None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.332854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.344010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.345232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.345552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.345780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.346451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:56.346768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.348148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.354027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.354655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.354967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.355240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.361292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 311/636] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:56.361292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.370557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.372054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a3cfa5fd-aced-48b8-822e-15a680297b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:56.386631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:56.387599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:56.389051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:56.389051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.389051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:56.389051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 326/637] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:56.389354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd934a-8acf-47a9-9d54-5640ad0877ee None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.389843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd934a-8acf-47a9-9d54-5640ad0877ee None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.389843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd934a-8acf-47a9-9d54-5640ad0877ee None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.391643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 327/638] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:56.396344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd934a-8acf-47a9-9d54-5640ad0877ee None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.470975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48534f-5a81-4d14-abb2-6a7d735001d7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.504908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.505521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.505801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.506077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.506760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:56.507069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.508428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.509259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd934a-8acf-47a9-9d54-5640ad0877ee None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.510946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.523595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 312/639] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:56.524528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.551270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.551894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.552165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.552433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.564471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.565908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-89fca5cb-0ca3-4454-a434-1efd72e44e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:56.584544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:56.585474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:56.586364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:56.586364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.586364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:56.587158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 328/640] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:56.587770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.588053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.588334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.609301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:56.611501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:56.629031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.635689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649bd79-ee9b-42c2-b123-1a5f9b6736a5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.669510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.670113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.670388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.670656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.671318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:56.671617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.673527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.691058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 313/641] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:56.691058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.697995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 329/642] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:56.706737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.803227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2640d1-8691-4bf3-860d-94f1f2fa58ab None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.838581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddd91-d5e2-4969-aacc-084cbab7c757 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:56.840382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.841037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.841324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.841596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.842246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:56.842544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.843979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.861463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 330/643] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:56.862340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.871069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.873988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:56.884820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:56.886184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:56.886632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:56.887127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:56.905924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:56.908340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:56.928878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:56.930095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:56.931265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:56.931265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:56.931265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:56.934865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 314/644] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:56.935724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:56.994884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe1d6-be0c-425a-bbd3-2d592761d733 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.033022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.033651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.033932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.034211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.040987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.042442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.043496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7515-3105-43d0-9908-017d3ea8ce7c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:57.054345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:57.055284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:57.061321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:57.061755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.061829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 331/645] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:57.061829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.083098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.083725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.084002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.084272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.092492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.094011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:57.105050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:57.105999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 315/646] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:57.107004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.136287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.173075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c492-6702-4088-ab2f-31d0ab091bff None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.211236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.211842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.212115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad8a9f-3f7e-4ab9-9286-730d24ae8986 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.212558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.212558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.213007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.213304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.215788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.230559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 332/647] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.231624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.253507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.254228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.254575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.254950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.265585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:57.271522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:57.272465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:57.273297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:57.277420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 316/648] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:57.278282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.312603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.338386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492702-a6fe-4c9e-b81d-6894453772e5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.367035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.376779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.377390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.377669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.377942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.378638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.378974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.380483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.386144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17f962-c5ca-4c36-a765-9b88ed7837b0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.392905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 333/649] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.393568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.428228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.428939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.429219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.429492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.430221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.430531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.432011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.450267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 317/650] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.455600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.505579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337db57-5b7e-4501-9794-25994178b4ad None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.543266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.543907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.544237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.544440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.548967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.553945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.555426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.562309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a541-9067-4854-866e-9b5f28f550b3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:57.567132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:57.568025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 334/651] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:57.574518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.608765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.609373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.609674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.610485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.611183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.611487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.612947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.622909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 318/652] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.623715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.631980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.692907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a104-c444-4753-9197-d20e445ade71 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.730018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.730623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.730747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019f688-38f0-4677-95a3-aadb9ddd6720 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.730944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.731215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.731943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.732249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.733666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.739407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 335/653] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.740178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.767852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.768482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.768764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.769035 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.789325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.845556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.847249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea6796-04ba-4080-b98a-3aadcc39e23b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.854233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8eca-b240-4eb2-9c29-90e5b92ab0b8 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:57.868536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:57.896221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:57.896851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.897134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.897406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.898151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:57.898458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:57.899935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:57.923057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 336/654] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:57.923057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:57.973507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 319/655] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:57.975201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4319c-ed76-40c3-b0a1-63603490ed7c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:57.975488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4319c-ed76-40c3-b0a1-63603490ed7c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:57.975761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4319c-ed76-40c3-b0a1-63603490ed7c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:57.977115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 320/656] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:57.978820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4319c-ed76-40c3-b0a1-63603490ed7c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.035371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.042528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c490e9e-7c58-4810-a7a2-51c45591777c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.077741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.078398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.079203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.079475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.088011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4319c-ed76-40c3-b0a1-63603490ed7c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.089990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.091541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:58.107238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.108389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:58.109453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:58.109453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.109453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:58.110261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 337/657] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.111182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.127230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.128084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.128362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.128631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.137383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.198541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.200022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.221167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9bc4f-2c45-4a7f-8e6e-813f80728d0d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.250840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.260058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.260687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.260962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.261230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.274029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.275489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ca412d19-22f3-4ece-b35b-dd0da4b3e666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:58.291044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.292023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:58.292929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:58.292929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.292929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:58.292929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 338/658] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.293648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.293648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.293897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.312182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 321/659] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:58.312182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.318673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:27:58.319964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:27:58.390228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 339/660] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:27:58.390934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.399779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.438128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a7234-5605-4aa3-b5b3-abd6328c53b6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.474910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.475837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.476281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.476704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.477486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:58.477937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.479554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.492412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 322/661] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:58.493518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.508904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.528026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31c9bb-6c69-41eb-a075-1fb1565a3d23 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.569635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.571049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.571503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.571924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.572716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:58.573166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.574748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.590599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 340/662] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:58.591805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.630877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4949a3-4bcf-49ff-ad25-401e59f52028 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.665663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.670100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.670854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.671212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.671592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.682437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.684028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:58.698400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.699533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:58.700637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:58.700637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:58.700637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:58.701855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 323/663] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.705342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.712503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9d06d-ea57-48c7-9d72-f9de52498574 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.752661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.753332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.753608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.753875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.761684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.763175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.772004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:27:58.774712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.775940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:27:58.777466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:58.780708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 341/664] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.781355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.816494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e26379-ac53-460a-bf3f-fde761ba7f69 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.849525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.850245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.850600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.850984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.857572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:58.859056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:58.871064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.872190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 324/665] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.873073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.898941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb89-a51d-4f14-ab51-d77fac0e86e5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:58.923312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:58.935331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:58.936091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:58.936460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:58.936831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:58.947951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:27:58.953173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:58.954856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:27:58.955896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:27:58.957460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 342/666] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:58.962953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:58.985962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce62c4-3504-4e57-823a-b79ca8c18f65 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.010548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.024539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.025165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.025435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.025711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.026352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.026656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.028263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.045047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 325/667] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.045047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.071951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923275f9-68d2-4f4d-9a12-d8adf95a7f14 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.105495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.106097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.106369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.106639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.107399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.107699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.109116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.126871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 343/668] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 24 07:27:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.127687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.151520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2ddc3-fd5b-4e42-b853-2fa0312707c6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.180897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.186825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.187625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.188693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.188969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.189664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.189967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.191425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.203492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 326/669] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.204990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.253082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.255581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa60c-052c-4ca7-84c1-62732e7ff6dd None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.292669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.293948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.294398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.294900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.295726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.296182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.298328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.306905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 344/670] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.307984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.316880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b6d83-fae0-45ae-a39d-09f9ffc824d4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.353285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.354657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.355329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.355894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.364580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.366643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:27:59.379213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:59.380339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:59.381448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 327/671] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:59.382131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.422379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312d7eb-d475-4348-9b03-0e44d218c935 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.425714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.460117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.460651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.460951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.461199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.491213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1cd5d-bb56-4c0f-a3ff-c6baeacb9b64 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.492879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.530311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.531111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.531458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.532350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.533142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=923067fd0c864ca9bb202ee9585062f3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.533450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.534891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.535158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.536841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.542736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 328/672] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/role_assignments?user.id=923067fd0c864ca9bb202ee9585062f3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.542736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.671328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afa2d1-ee0d-46e4-bc58-04794ff5242a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.684684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 345/673] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:27:59.685650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.689756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.715575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.716901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.717367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.717825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.730198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.732212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:59.754584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:59.755617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:59.756507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:59.756507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:59.756507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:59.756507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 329/674] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:59.756507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.777171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.800996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45422280-41e9-4050-ad10-a8c9ce81414b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.837879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.838486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.839162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.839162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.839737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:27:59.840041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.841463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e9be-6166-40fe-b8ab-041657dab0ac None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 346/675] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54202d86-3789-415b-a0dd-bd619bcfbc14 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54202d86-3789-415b-a0dd-bd619bcfbc14 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54202d86-3789-415b-a0dd-bd619bcfbc14 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 347/676] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:27:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:27:59.858197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54202d86-3789-415b-a0dd-bd619bcfbc14 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.873634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343137-b7f0-4f08-87a2-6b342e0bdce5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:27:59.906487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:27:59.907755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:27:59.908100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:27:59.908460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:27:59.918874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:27:59.920300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:27:59.928253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:27:59.935161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:27:59.936339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:27:59.937500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:27:59.937500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:27:59.937500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:27:59.937500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 330/677] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:27:59.937500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:27:59.975533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54202d86-3789-415b-a0dd-bd619bcfbc14 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.011230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.011846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.012164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.012446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.013089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:00.013387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.014780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba300a0f-7a4c-4887-88fc-b63b389522de None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.015797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.033122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 348/678] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:00.033122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.033122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.033122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.048923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:00.050157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:00.058842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6145e-7201-48c3-b61d-2bee32fa17f2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.096035 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.096948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.097389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.097671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.098390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=ea288fe3842541999821bd863ca9d0c2) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:00.098741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.100143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.106086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 331/679] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:27:59 2022] GET /identity/v3/role_assignments?user.id=ea288fe3842541999821bd863ca9d0c2&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:00.106909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.143098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 349/680] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:28:00.144366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.269783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a11eb-bbcc-4bfb-8183-16ac13e7e882 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.283316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.316681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.317476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.317883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.318159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.319031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:00.319339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.321040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.322606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.337880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 332/681] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:00.339526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702adeff-5c88-4c20-816b-f676705bced9 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.344768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.393409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.394048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.394330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.394662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] PATH_INFO: `/v3/users/swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.402343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.403943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:00.417375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:00.418336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:00.419321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:00.422822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 350/682] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:00.424545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.476481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ea80e-35ba-473d-9a12-25c9d08c7c26 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.512088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.512719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.512996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.513263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.520752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.522138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:00.534791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:00.535903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 333/683] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:00.537099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.559930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2de94-75a7-4fea-b540-e09d46cb10da None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.582840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.605560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.611712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.612334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.612611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.612876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.613615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:00.614033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.615580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.637333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 351/684] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:00.638954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.661215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19fcb-c81d-44d1-b523-285a9ce5ee5d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.712297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.713441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.713830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.714204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.728702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:00.735817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:00.736965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:00.738017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:00.740529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 334/685] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:00.741494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.767516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4a43a-e15e-43de-9d14-6118e5cbda43 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.814027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.815127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.815754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.816137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.817229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:00.817639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.819913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.839494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 352/686] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:00.840908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:00.867585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d419cc-f0c6-4d0b-810b-cc7b50b37248 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:00.883137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.895427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:00.911843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:00.912456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:00.912760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:00.913010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:00.989294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:00.991148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:00.995648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ae2fa-3df3-45bf-a738-e755c19ab358 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.050972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.051792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.052189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.052474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.053162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:01.053468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.055630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.068744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 353/687] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:01.069877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.114794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 335/688] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:28:01.116090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.137352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.178312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.186939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178db33-0efd-4b68-a8a6-3ac4ff002dad None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.234041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.235278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.235581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.235858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.237028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=ae3cfe87a8b347729eb0cbb4f718d394) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:01.237475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.239340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.245991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc7090-ab04-466d-875f-05b2da33d6a7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.247570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 354/689] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/role_assignments?user.id=ae3cfe87a8b347729eb0cbb4f718d394&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:01.248740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.286378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.287773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.288261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.288783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.301113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.303013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:01.319966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:01.320980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:01.321864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:01.321864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:01.321864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:01.324224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 336/690] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:01.331986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.430289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270053a-0ade-43fa-b139-940ada5be450 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.467602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.467917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.469600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.470388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.470982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.482764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.484389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:01.501258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:01.502369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:01.503591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:01.503591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:01.503591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:01.503591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 355/691] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:01.508856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.515244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559604-96b0-4c70-a6f1-10921639d82b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.526013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.570978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.571946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.573143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.573519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.574274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:01.574979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.577110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a034-0f90-4524-8661-306d746fb654 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.596397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 337/692] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:01.597934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9881b-54dd-4bc8-b16f-3c0f081b2c8f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.598322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9881b-54dd-4bc8-b16f-3c0f081b2c8f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.599182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9881b-54dd-4bc8-b16f-3c0f081b2c8f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.600856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 338/693] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:01.601709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9881b-54dd-4bc8-b16f-3c0f081b2c8f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.640034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeec0d3-48e1-4ee5-85ff-b8c218b88958 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.687338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.688528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.688867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.689149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.689805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:01.690110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.692080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e3c7f-3103-494e-81ec-d07e634a0662 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.709556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 356/694] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:01.711287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.711573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.712279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.722442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9881b-54dd-4bc8-b16f-3c0f081b2c8f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.738160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:01.740074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:01.754043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.763890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.764776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.765146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.765544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.774009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.775823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:01.788674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:01.789839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:01.791021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:01.797833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 339/695] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:01.798811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.824530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:01.832784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 357/696] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:28:01.834173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:01.912543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba09a1-780e-4e26-b597-adefd517b9ce None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:01.949582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:01.950451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:01.950858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:01.951271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:01.959384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:01.960689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:01.974648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:01.975816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:01.976941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:01.978150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 340/697] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:01.983639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.023251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afafcb9-003f-4f83-9977-60654df5d8f1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.080221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.103480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.104290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.104575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.104850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.105605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:02.105909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.107413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.138354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 358/698] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:02.145385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.147147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b37f8-dcfe-45c4-95dd-5251e2a9ce6c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.216045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.217055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.217574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.219628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.222968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:02.223309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.224785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.242765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 341/699] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:02.245028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.255076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.302969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e8f2-aa61-483c-93bb-036d74488685 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.348343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.349710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.350585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.351095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.366569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecf994-071f-4085-b275-84682256a15a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.374383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.403579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.404516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.405158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.405469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.406618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.408574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:02.419721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:02.420774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:02.423965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 342/700] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:02.424742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.441005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.442850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.507322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.530917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad989-f2f6-4093-bbcb-0ff21616427e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.557263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 359/701] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:28:02.558387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.573549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.574232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.574515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.574823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.575549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=5c30642f91c54258b4ceec9005408be3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:02.575855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.577333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d94c48-85e9-44c4-a74f-15e4cc3b0064 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.590830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 343/702] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/role_assignments?user.id=5c30642f91c54258b4ceec9005408be3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:02.592123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e686-9785-4909-b8ce-bd5ec145448e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.592404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e686-9785-4909-b8ce-bd5ec145448e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.592671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e686-9785-4909-b8ce-bd5ec145448e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.598973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 344/703] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:02.598973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e686-9785-4909-b8ce-bd5ec145448e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.639194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.716353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37878299-8366-4e4c-8f7f-81e05934c12a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.721281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e686-9785-4909-b8ce-bd5ec145448e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:02.760604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.761372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.761653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.761923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.762552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:02.762941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.763642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:02.764311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e206c-0128-440c-b18d-414fb878f2e8 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.764620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.765038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.765623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.770271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 360/704] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:02.771647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:02.771838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:02.772123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:02.781174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:02.783578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:02.794145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:02.795807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:02.801848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:02.802865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:02.803789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:02.803789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:02.803789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:02.804788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 345/705] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:02.804788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.817302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.885795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 361/706] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:28:02.886722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:02.938726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:02.960415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171021-13a8-4ad9-8f67-d464dd22f527 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.017759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.018399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.018678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.018978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.019614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:03.019913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.021305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.038907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 346/707] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:03.071863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90adb1-6d28-4f9e-b175-b06a777928de None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.128167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.128794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.129075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.129346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.130008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:03.130312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.131731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.150101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 362/708] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:03.155360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.193198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.211744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:03.311598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86de17-44f5-4206-b786-b8eec899c3fb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.339378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85fcee-e08c-4465-bfdc-6352f390a1ef None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.349610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.350496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.351015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.351500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.360452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.362169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:03.374622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:03.376032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:03.376894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:03.378472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 347/709] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:03.384642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.390594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.391827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.392111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.392385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.406455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:03.410827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.412688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:03.441406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:03.442610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:03.443637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:03.443637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:03.443637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:03.443961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 363/710] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:03.452790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.563829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77978b6e-610c-4d9b-8ba0-55285dc170b6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.591923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:03.617802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.618811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.619240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.619662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.620519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:03.620982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.624297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.649782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 348/711] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:03.658852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.670523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318af0-4bf5-4fb8-8732-3517957edd8c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.708133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.709187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.709649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.710100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.711058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:03.711560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.713295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b75fd-d7ea-43d2-bc42-dd33d0320dd0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.728183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 364/712] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:03.729652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f4c1-3d6e-4a16-9b78-d65aeb1755a9 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.729652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f4c1-3d6e-4a16-9b78-d65aeb1755a9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.729652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f4c1-3d6e-4a16-9b78-d65aeb1755a9 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.734802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 365/713] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:03.736533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.736822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.737096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.742219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:03.762661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:03.763902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:03.779562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217621-342c-4fa8-b275-28678ae5e692 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:03.822803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:03.823724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:03.824284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:03.824564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:03.825298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:03.825602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:03.827640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:03.835513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 349/714] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:03 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:03.836316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:03.854749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 366/715] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:28:03.966929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ff0de-3e28-439c-96ec-17679bc1c57f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:04.012492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:04.013126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.013409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.013684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.014350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:04.015843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:04.027584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:04.055707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 350/716] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:28:03 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:04.204152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:04.270332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:04.319617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f30d2-7864-4e84-9649-945f77d90902 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:04.361971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:04.362637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.362941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.363211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] PATH_INFO: `/v3/services/network` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.363809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:04.365193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:04.378072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:04.380066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:04.381332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 367/717] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:04 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:04.387776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:04.499973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a38d7-fdc3-44d4-8a43-674f27c944b8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:04.540816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:04.541434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.541713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.541983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.542617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] RBAC: Adding query filter params (name=network) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:04.542972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:04.544348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adc96d-2bd0-449e-8a69-3b4a62fbd456 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:04.549667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 351/718] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:04.554835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:04.715156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4801-4ed7-4c41-94ce-2c5f0b2f9a97 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:04.738012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:04.759721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:04.760336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.760614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.760886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.761528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] RBAC: Adding query filter params (type=network) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:04.761833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:04.763256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1257c1e-ff53-437b-b087-cde9deffc725 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:04.771626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 368/719] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:04.897332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd396-8ae4-4e89-93f7-c89f13d92a57 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.897691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd396-8ae4-4e89-93f7-c89f13d92a57 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.897906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd396-8ae4-4e89-93f7-c89f13d92a57 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.899589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 352/720] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:04.905131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad527-b91e-409c-aab3-4c69e159e3de None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:04.905414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad527-b91e-409c-aab3-4c69e159e3de None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:04.905688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad527-b91e-409c-aab3-4c69e159e3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:04.932411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-192ad527-b91e-409c-aab3-4c69e159e3de None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:04.933624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-192ad527-b91e-409c-aab3-4c69e159e3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:05.019817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 369/721] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:28:05.318720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dd396-8ae4-4e89-93f7-c89f13d92a57 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:05.353440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c50d7-9eb7-45f4-9ef8-fe5665a3b5cc None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:05.353953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c50d7-9eb7-45f4-9ef8-fe5665a3b5cc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:05.354063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c50d7-9eb7-45f4-9ef8-fe5665a3b5cc None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:05.355611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 370/722] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:05.360738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:05.361024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:05.361305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:05.390675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:05.392436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:05.415514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:05.499389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dd396-8ae4-4e89-93f7-c89f13d92a57 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:05.503281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 371/723] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:28:05.546576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:05.547342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:05.547615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:05.547884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:05.558524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:05.563539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:05.564590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:05.565704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 353/724] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:05.572472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:05.645280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:05.717979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10805c5d-3eda-42b2-8646-0f60c60a89be None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:05.752986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:05.753596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:05.753877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:05.754147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:05.754855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:05.755156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:05.756514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:05.765060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 372/725] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:05.771143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:05.834846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:05.892449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a57c1-06ee-4a94-ab57-d907cd1a403f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:05.908699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:05.931013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:05.931792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:05.932163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:05.932528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:05.943634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:05.945323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:05.960173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:05.961324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:05.962438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:05.962438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:05.962438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:05.971027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 354/726] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:05.976963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:06.000964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b79d42-65af-4c47-bbdb-7e1704ff8250 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:06.013216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:06.099451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:06.100085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.100371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.100650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.101325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:06.102739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:06.116911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cfbb0-eefc-4045-a3f8-d00aee38dcf6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:06.146556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 373/727] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:05 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:28:06.179858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:06.180493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.180774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.181053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.181819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:06.182126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:06.183599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2c91d-1273-4977-827f-a66ac7295e30 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:06.202723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 355/728] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:06.209552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:06.263158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1044bae-2651-47f3-9804-53f989a6aa1b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.263521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1044bae-2651-47f3-9804-53f989a6aa1b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.263713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1044bae-2651-47f3-9804-53f989a6aa1b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.265138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 356/729] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:06.271402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.271681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.271947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.302645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:06.304513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:06.344291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:06.364655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c89a7-c881-4c1b-9284-df082bd6427a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:06.408666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:06.409651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.410165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.410640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.419229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:06.421006 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:06.424614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 357/730] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:06.436857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:06.437886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:06.439550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:06.440350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 374/731] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:06.446770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:06.549859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213058-6bef-476e-b3ae-f42f3cefce7a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:06.587460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:06.588073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.588349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.588620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.589354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:06.589658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:06.594631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc02de-8359-4644-8ac3-0f2b9d329589 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:06.614090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 358/732] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:28:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:06.619930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:06.623370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe54fe-50b6-4ec9-917c-d1535a70e8a9 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.623872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe54fe-50b6-4ec9-917c-d1535a70e8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.624365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe54fe-50b6-4ec9-917c-d1535a70e8a9 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.626798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 359/733] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:06.631515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.631800 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.632073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.671083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:06.763458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244f691-bec9-470b-ab8a-01f5174cdbe1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:06.808916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:06.809560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:06.809840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:06.810117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:06.860507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:06.862044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:06.865866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:06.893192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:06.894676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:06.922947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:06.948496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 360/734] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:28:06.950272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.013324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 375/735] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:28:07.019265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.081991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a491bd-2068-4c2b-bc67-1a4113a8b812 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.126720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.129405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.130194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.130474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.132931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892d400-1a35-49b6-95dc-4c5f3ba25eb2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.152395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.155527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:07.160276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:07.161207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:07.162590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 361/736] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:07.169857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.169982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.184816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.185453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.185721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.185993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.198031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.199517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:07.229560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:07.230537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:07.231448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:07.231448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:07.231448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:07.232965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 376/737] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:07.238751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.330594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7f81c-8e88-41f7-ae12-f6250435c0b0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.368558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.369297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.369580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.369852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.370626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:07.370953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.372482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.382214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 362/738] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:07.387350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.390460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d9c18-28b8-492e-aeba-72cb27ffef03 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.428449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.428977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.429259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.429535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.430179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:07.430483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.431899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.445477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 377/739] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:07.446215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.456934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.482238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.504258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c22b-7b37-41cb-aab2-2e5d70e81cd9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.539531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af232b-4d29-4424-8333-e82877260844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.540159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af232b-4d29-4424-8333-e82877260844 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.540433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af232b-4d29-4424-8333-e82877260844 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.540696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af232b-4d29-4424-8333-e82877260844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.552814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af232b-4d29-4424-8333-e82877260844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:07.555316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ab00-b9e1-4d6d-89a3-19a954565f0a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-64af232b-4d29-4424-8333-e82877260844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:07.558432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:07.559456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 363/740] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:07.560296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35470b8-aff1-49d0-8064-20f55d5f9165 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.561074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35470b8-aff1-49d0-8064-20f55d5f9165 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.561074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35470b8-aff1-49d0-8064-20f55d5f9165 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.562333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 364/741] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:07.567941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35470b8-aff1-49d0-8064-20f55d5f9165 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.604138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.604769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.605105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.605371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.615328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.616770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:07.631527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:07.632464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:07.633349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:07.633349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:07.633349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:07.633349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 378/742] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:07.633349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.684697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35470b8-aff1-49d0-8064-20f55d5f9165 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.709414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.721592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.722221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.722502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.722803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.729713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.731183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.731410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-24b7ca1a-8609-4328-b36a-07d42b7298a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:07.743417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:07.744297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:07.745173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:07.745593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3d901-0ffb-46c0-854f-55c5b1658b5a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.749332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 365/743] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:07.750488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.750796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.751071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.768914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:07.770086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:07.781240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.782602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.783086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.783526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.784339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:07.784801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.786325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.795697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 379/744] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:07.796691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.845449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 366/745] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:28:07.851571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:07.924848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c349-254c-4766-a778-a478156cfd8c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.963480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007f0a-e419-43d8-a5bd-1f6fac13f0a0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:07.970146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:07.971013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:07.971292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:07.971564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:07.972210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:07.972514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:07.973891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:07.983129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:07.990766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 380/746] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:07.990766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.009121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.014644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.015384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.015735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.016219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.017039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:08.017450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.019932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.033940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 367/747] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:08.034936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.114168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7083f-8594-4317-9842-fb1f1b899e69 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.153257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.153886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.154164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.154678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.158078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356edc3c-6506-455f-a86f-7fd0ff31db62 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.165404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.166925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:08.182191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:08.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:08.184062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:08.184062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:08.184062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:08.184062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 381/748] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:08.184869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.199496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.200135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.200415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.200685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.208243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.209677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:08.221563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:08.222497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 368/749] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:08.224041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.255082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.256412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.299239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c0e78-cdc7-4203-a9c4-3ee6d7c3d52c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.331418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f095711-413f-4aee-b82a-f33b95ebc661 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.336277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.336882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.337165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.337439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.338164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=5c30642f91c54258b4ceec9005408be3) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:08.338820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.339991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.354736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 382/750] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/role_assignments?user.id=5c30642f91c54258b4ceec9005408be3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:08.354736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.367401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.368023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.368306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.368584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.369246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.370672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.394771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 369/751] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:08 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:08.394771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.492946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b3a7-1143-4719-b359-a8a1cc19e6e3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.515213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.523534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.529495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70484b-f239-470f-8ef8-5316224db99a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.535206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.535831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.536106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.536375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.537029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:08.537327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.538809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdaf2c-03fa-4c56-acf8-89714458a91d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.553705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 383/752] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:08.555045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505804-e875-4a03-9140-7125e32647b8 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.555331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505804-e875-4a03-9140-7125e32647b8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.555601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505804-e875-4a03-9140-7125e32647b8 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.562057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 384/753] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:08.562057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505804-e875-4a03-9140-7125e32647b8 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.577209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.577973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.578364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.579320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.583629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:08.583629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.583629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca7c-cdf4-4045-9c17-df6a2ddc8c7d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.602748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 370/754] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:08.603869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.604156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.604427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.628020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:08.629318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:08.689268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505804-e875-4a03-9140-7125e32647b8 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.702734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 371/755] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:08.704227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.767502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.768060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.768337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.768607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.779256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.780702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.795025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:08.804159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:08.805518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:08.807001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 385/756] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:08.815664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.821337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04e3c1-42de-41d8-9162-274d267c9ff7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:08.857916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:08.858549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:08.858854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:08.859114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:08.870494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:08.872184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:08.888687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:08.889687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:08.890562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:08.890562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:08.890562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:08.890562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 372/757] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:08.895729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:08.952179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.003198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df99a-e789-4a1c-a317-73e5eebfed42 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.035125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.043408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.044877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.045675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.046632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.048824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:09.049828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.052853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.061127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9de8e-7fe8-4b28-8afa-c07c3c32c427 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.067057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 373/758] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:09.072424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.099128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.099740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.100015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.100283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.101155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:09.101456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.103013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.123183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 386/759] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:09.124171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.189839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91885f-540e-46a6-8285-7cb59661e229 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.236334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23bd1-e131-4016-a714-afe3ba6f0987 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.239563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.240670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.241532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.242018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.242498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.243327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.245348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:09.258671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:09.259702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:09.260519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:09.263582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 374/760] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:09.264773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.276743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.277270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.277687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.278066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.285924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.287539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:09.299831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:09.300847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:09.301887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:09.307853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 387/761] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:09.308842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.334157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.400094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9d7b-20a1-42b8-9e6a-fd3923d9c873 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.424015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba121bd5-f3a5-40e5-ad3e-de8b7b5b0a4a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.444356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.445167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.445557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.446161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.469961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.470824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.471229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.471608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.472452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:09.472865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.474414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.480196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 388/762] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:09.481172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.512462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.521989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.524147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.588283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.606038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b625c-5437-4a29-89f1-58422bd0f7ca None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.636086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 375/763] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:09.637276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.650146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.650888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.651221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.651493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.652232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:09.652586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.654339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.671608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 389/764] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:09.672556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.742388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8776b1-570c-48fc-99ee-357087adcfef None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.773420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.779490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.780398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.780904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.781386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.782783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:09.783295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.784878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.788065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6e36-fd15-430c-be4a-75962f4156e9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.790639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 376/765] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:09.791748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.810169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:09.826580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.827271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.827532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.827805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.839713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.841152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:09.856565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:09.857599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:09.858477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:09.858477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:09.858477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:09.862302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 390/766] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:09.863163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.900396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef99e-52a2-4eb1-8755-f39722ad02dd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:09.936765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:09.937810 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:09.938302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:09.939079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:09.941152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=cff82f8d12d842d1baab39bd73ff54cd, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:09.942756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:09.966726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 377/767] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:09 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:28:09.968328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:09.975707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd49a3-0559-4ecf-a5e6-91f7f9ae7ec9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.053944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.054609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.054920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.055231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.059471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] RBAC: Adding query filter params (interface=public, service_id=c85b12d16c74452da8ba1420d69c71e8, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:10.059828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.061302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.073329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 391/768] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/endpoints?service_id=c85b12d16c74452da8ba1420d69c71e8&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:10.083775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:10.087781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:10.119231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:10.154995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629720e-a535-44c9-bf59-e8d443e832b3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.194672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.195477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.195875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.196248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.197007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:10.197958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.198900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.212137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 378/769] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:10.220139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:10.236728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1a353-e785-469e-a6c8-1ccd5f39c56d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.273702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.274433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.274736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.276073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.276786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.278339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.286679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 392/770] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:10 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:10.342754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9d6c-57d9-4a33-907f-98604b39f89a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.402130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.402725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.403007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.403282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.412061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:10.417329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.418845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-73706c06-b4f5-4a75-8d8b-294db9c418fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:10.432349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:10.433360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:10.434246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 379/771] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:10.440646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:10.473872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f786-d172-4389-a720-adf1c4753f50 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.474195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f786-d172-4389-a720-adf1c4753f50 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.474583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91f786-d172-4389-a720-adf1c4753f50 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.476445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 380/772] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:10.488935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e840e-003d-4ae6-8c40-6ba4b744a5f7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.489508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e840e-003d-4ae6-8c40-6ba4b744a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.490023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e840e-003d-4ae6-8c40-6ba4b744a5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.509973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-009e840e-003d-4ae6-8c40-6ba4b744a5f7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:10.511537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-009e840e-003d-4ae6-8c40-6ba4b744a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:10.528532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:10.570217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954afc9-7a9c-414f-a6e4-2b0f6aeeb0e1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.624624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.625239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.626179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.626470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.627206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:10.627532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.628882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 381/773] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:10.629299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.630012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0b07-00df-4d5c-b087-9745d5bd48fd None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.630297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0b07-00df-4d5c-b087-9745d5bd48fd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.630643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0b07-00df-4d5c-b087-9745d5bd48fd None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.637772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 382/774] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:10.643225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 393/775] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:10.643628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.643974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.644252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.654014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:10.662720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:10.664816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:10.780204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 383/776] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:10.792699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeba5f-7e89-43e4-9f91-75eb01e973d9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:10.825312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:10.829166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:10.829780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:10.830270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:10.830736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:10.831474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=c1d8abb0b797488c8d07db05d6cc8892) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:10.831989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:10.833416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:10.839351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 394/777] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:10 2022] GET /identity/v3/role_assignments?user.id=c1d8abb0b797488c8d07db05d6cc8892&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:11.018258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.084641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:11.121556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.135608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fb42e-d3c9-490b-a0cd-703041cf7a89 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.171141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.172047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.172366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.183630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:11.189768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:11.190682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:11.191538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:11.194797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 384/778] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:11.197251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.287984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33378c-0e1b-4c17-b36c-7bc970086af6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.325422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.326057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.326324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.326598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.327218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.329601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:11.339073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:11.340053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:11.340889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:11.341621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 395/779] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:11.348078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.415501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:11.446976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f637-adcf-48ba-a558-a0f303476b5d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.478069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546e65a-668f-4baa-bd0b-37866481fbe0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.484729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:11.519628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.520241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.520515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.520781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.521449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:11.521744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.523143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.542587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 385/780] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:11.552026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.555963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.558315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.558594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.558893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.559533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:11.562117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.564328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.576114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 396/781] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:11.590910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.707588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821471-494f-4e0b-b91f-cfe0fb2118e4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.759593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.760197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.760473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.760743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.761368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:11.761728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.763131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.767034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:11.769596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 397/782] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:11.776255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.784150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf4a3d-cdf1-477f-98ad-82cbe618111c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.884196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113604e-16ae-465f-bdad-28cb5bb9acf2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:11.916049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.916809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.917090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.917362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.919943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:11.920552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:11.920834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:11.921107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:11.921811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] RBAC: Adding query filter params (interface=public, service_id=d523668d1c244a658e126d9f38f693c3, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:11.922119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.925901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.936772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 398/783] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/endpoints?service_id=d523668d1c244a658e126d9f38f693c3&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:11.939439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:11.944145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:11.945608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:11.975213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:11.976260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:11.977142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:11.977142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:11.977142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:11.990070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 386/784] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:12.007886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:12.051049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:12.061476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:12.109478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176e0b-036e-4873-aa20-cb6735db1459 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:12.145978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:12.146965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.147495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.147983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.148862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:12.150621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:12.161752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 399/785] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:11 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:12.184927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884eb3-3724-4c50-8730-2c8b8dddb376 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:12.233154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:12.233774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.234051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.234322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.235542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:12.235843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:12.237224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d78d8-cc7a-4217-a3e8-04727ea1a3a4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:12.249872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 387/786] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:12.256147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:12.348350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939f9a-5c47-4441-bb0c-c4c13c4d7a15 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.348669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939f9a-5c47-4441-bb0c-c4c13c4d7a15 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.348904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939f9a-5c47-4441-bb0c-c4c13c4d7a15 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.350364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 388/787] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:12.355513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.356029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.356501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.378286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:12.380028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:12.439334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9d25-a167-4bc1-9754-994d5e460d2c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:12.453962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 389/788] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:12.466882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:12.506683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:12.507345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.507626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.507902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.523755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:12.525213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-e10a0d2c-3fa9-4594-b83a-84bf80476d9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:12.540963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:12.541925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:12.542890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 400/789] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:12.548410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:12.590723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:12.659497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529c68d-e526-47b1-9027-ce6c9c0103a5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.660010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529c68d-e526-47b1-9027-ce6c9c0103a5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.660010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529c68d-e526-47b1-9027-ce6c9c0103a5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.666727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 401/790] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:12.666875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.667216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.667482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.704164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d81f5-1bb8-4be4-be26-0a52d115ac74 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:12.715081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:12.716271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:12.767062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:12.767679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:12.767955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:12.768346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:12.769090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:12.769393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:12.771075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:12.804028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 390/791] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:28:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:12.808872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 402/792] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:12.810154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:12.827824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:12.930501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:12.956653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf78156-9275-4251-ab58-26a3ed4dbf6a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.005507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.006877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.007164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.007479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.015214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c04a7-b408-4c77-a4ff-8015a3dd6d50 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.019024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.030764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.032831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:13.059345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:13.060348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:13.061233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:13.061233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:13.061233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:13.061865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 391/793] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:13.078097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.082240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.083114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.083571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.084040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.095468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:13.100668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:13.101573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:13.102384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:13.105370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 403/794] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:13.115975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.197714 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44ccb-cd4e-4097-9590-57e6ff158979 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.218550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d85419-d7eb-40f4-8512-9706918456fc None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.227271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.233619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.234229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.234503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.235444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.236887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:13.237222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.238568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.254726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 392/795] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:13.254726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.261777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.262387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.262664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.262967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.263631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:13.263930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.265300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d750-d023-4aef-9b29-f8450d941f9a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.275053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 404/796] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:13.280458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1e049-056a-4b25-bb84-52bf2b67e70a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.280912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1e049-056a-4b25-bb84-52bf2b67e70a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.281194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1e049-056a-4b25-bb84-52bf2b67e70a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.286956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 405/797] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:13.286956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1e049-056a-4b25-bb84-52bf2b67e70a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.311246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.358545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6766-d448-4eed-bc6e-ae360bfd358a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.398664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.398885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1e049-056a-4b25-bb84-52bf2b67e70a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.399334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.399612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.399884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.400724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.403038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:13.415201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:13.416194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:13.418826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 393/798] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:13.419653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.434571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.435239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.435522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.435794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.442935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.444365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-4b620725-5f7b-45bb-be2b-1990eb44049d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:13.455515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:13.456422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:13.457388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 406/799] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:13.457871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.458273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.458367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.478465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:13.479563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:13.492426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.543110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accde656-2423-4580-b8b7-162d356a04f0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.561531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 407/800] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:13.562488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.586637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.590708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.591363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.591640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.591909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.611062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.613074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:13.629809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:13.631119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:13.632014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:13.632014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:13.632014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:13.634044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 394/801] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:13.635141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.752905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29d806-a5e0-44c7-a0b8-fbd475b17e5c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.760420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.788688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.789297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.789572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.789840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.790530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:13.790882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.792305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.805015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 395/802] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:13.805909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.834533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e663-ef60-46a7-8109-cb18de97cf1f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.874064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.876065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.876357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.876641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.877297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:13.877599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.879009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.883782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 408/803] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:13.887823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.909799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b922b9-ddea-4665-a605-2ed641227f23 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:13.948910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:13.949843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:13.950305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:13.950791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:13.951685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:13.952183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:13.954373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:13.967403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 396/804] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:13.968511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:13.984490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:13.998354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.008420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eff35-b306-481d-aed6-90702d358e5f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.044913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.045564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.045840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.046109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/5c30642f91c54258b4ceec9005408be3/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.048091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=5c30642f91c54258b4ceec9005408be3, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.049515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.064474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 409/805] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:13 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/5c30642f91c54258b4ceec9005408be3/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:28:14.065174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.089600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21cb75-e92f-44af-9efe-b9c4b7dae348 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.133025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.133798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.134163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.134541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.150913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:14.158673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:14.159703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:14.160542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:14.164905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 397/806] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:28:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:14.165895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.204941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc08d3-29ee-47c3-b498-6904f4dc2a6f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.255764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.256754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.257250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.257703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.258590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:14.258915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.260538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.265834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 410/807] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:14.266729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.269341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.279580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.296728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd83674-481a-4523-bdcf-e08fc9893a0a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.337519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.338403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.338861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.339305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.348122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.349779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:14.362813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:14.363934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:14.365070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:14.367164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 398/808] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:14.368276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.380148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c09f5-9e3e-4d4e-9b06-2e9992a0c385 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.416473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.417219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.417567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.417906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.418641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:14.419038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.420459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acb326-1cda-46c8-bfd4-0d20159dc6a3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 411/809] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6bd73-21b8-4ff4-98ec-4dda2926f54a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6bd73-21b8-4ff4-98ec-4dda2926f54a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6bd73-21b8-4ff4-98ec-4dda2926f54a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 412/810] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:14.433536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6bd73-21b8-4ff4-98ec-4dda2926f54a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.483993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd26a-f162-4f29-ba3c-020f0b92bd79 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.502567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.522071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.522599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.522902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.523197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.523872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.525263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.544402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6bd73-21b8-4ff4-98ec-4dda2926f54a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.548032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 399/811] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:14.548502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.551390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.599833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.600667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.601045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.601482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.602333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:14.602912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.604533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c402c-143f-4ad2-9808-9551885d4906 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.628449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 413/812] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:14.629872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904723-8136-4e94-a515-f8d521112979 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.630255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904723-8136-4e94-a515-f8d521112979 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.630962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904723-8136-4e94-a515-f8d521112979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.651470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-77904723-8136-4e94-a515-f8d521112979 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:14.652907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-77904723-8136-4e94-a515-f8d521112979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:14.705352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb122b-c5e5-4512-af25-604413dc191d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.738436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 414/813] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:14.739400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77904723-8136-4e94-a515-f8d521112979 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.768628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.769436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.769789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.770156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.790623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.792278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:14.795988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:14.811343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:14.812862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:14.814873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:14.814873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:14.814873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:14.815528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 400/814] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:14.821434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:14.856328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77904723-8136-4e94-a515-f8d521112979 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.892023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:14.892731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:14.893013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:14.893282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:14.916434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:14.973636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:14.982682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2385a1-c35d-4c68-bc33-7b87a0932009 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:14.983216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.034182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.048566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.049470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.050015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.050293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.050967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:15.051268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.052811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.068437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 401/815] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:15.069607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.096127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 415/816] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:15.096981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.193500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacedc74-ffb9-43d3-8166-5a4d0a00c75c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.214650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.218489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08267e1-e2ea-4b56-9e70-16e6809817e6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.243159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.243798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.244128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.244482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] PATH_INFO: `/v3/services/network` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.245076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.246636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:15.258544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:15.259613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:28:15.260441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:15.263503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 402/817] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:15.265155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.270557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.271259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.271538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.271807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.279970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.281415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:15.295755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:15.296710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:15.297690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:15.302151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 416/818] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:15.302982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.317977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.389613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946d53-97af-4e68-8c71-b062af81baaf None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.433121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64556-f040-4246-954f-36159a45e5ac None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.433452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.434095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.434380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.434654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.450206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.452219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.471457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:15.472278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:15.473273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:15.474157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:15.474157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:15.474157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:15.474836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.475324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.475792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.476702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] RBAC: Adding query filter params (name=network) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:15.477205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.478232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 403/819] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:15.479042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.479151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.489254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 417/820] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:15.490334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.519304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.568399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.612822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8535ba-bee0-4da4-b8dc-38fa4d2b50af None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.621126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb76-a19d-4417-803d-f5db03b651e8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.650834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.651427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.651707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.651981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.652632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:15.652937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.654518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.667749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.668744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.669249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.669722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.670663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:15.671193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.672631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 418/821] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:15.673165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.673545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.701852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 404/822] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:15.701852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.795396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4fd13-7a85-44c7-bee0-6e7f5c9b25cb None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.811595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.826719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd7e93-3237-4a39-a2ad-ecb3a3efcf2f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:15.838276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:15.844869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.845897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.846325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.846940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.848011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] RBAC: Adding query filter params (type=network) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:15.848478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.849991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.861486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 419/823] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:15.861486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.873802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:15.874618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:15.874998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:15.875460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:15.884372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:15.886221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:15.899037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:15.900147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:15.901236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:15.902564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 405/824] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:15.903794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:15.988855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93136a-b184-4229-b059-1ae7cb938325 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.018985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c10e87-f9c7-4bf8-9206-b3b94fa8ecb0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.032901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.033834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.034256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.034825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.047803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.050155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.056982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.057830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.058218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.059015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.059527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.061090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:16.068517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:16.069646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:16.070759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:16.070759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:16.070759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:16.077138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 420/825] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:16.077138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.098723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 406/826] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:15 2022] POST /identity/v3/endpoints => generated 337 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:16.098723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.098723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.149468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.209377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf9d32-163a-4eb2-9512-88578bb1db11 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.229371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41699a7-9b52-44ff-baa3-e1a776d5dbde None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.247966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.248973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.249455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.249927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.251395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:16.251920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.253595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4142b-7cf4-4deb-bebd-59a60fd82d79 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.267425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 421/827] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:16.269187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eecb2-d52d-47ac-8141-ca118081fe2c None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.269691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eecb2-d52d-47ac-8141-ca118081fe2c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.270258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eecb2-d52d-47ac-8141-ca118081fe2c None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.270865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.271482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.271754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.272052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.272668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:16.272858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 422/828] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:16.272968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.273835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eecb2-d52d-47ac-8141-ca118081fe2c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.274324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc044b-9b50-4ba3-853e-c4364028a09a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.288107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 407/829] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:16.289358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.289641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.289936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.318057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:16.319301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:16.387512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.398970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eecb2-d52d-47ac-8141-ca118081fe2c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.400916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 408/830] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:16.401874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.415486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.449833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.450527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.450889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.451267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.452067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:16.452442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.453969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.466838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 423/831] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:16.528091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb35c14-b4dd-43df-86af-e0f5cb39455e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.568825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.569731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.570198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.570626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.579539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.581260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:16.599360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:16.600362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:16.601385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 409/832] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:16.601859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.696884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.726879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a372e1-480b-4193-89f6-4c99165f1e2d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.752807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.770469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.771109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.771388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.771663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.772354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:16.772659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.774092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5ad8-73ad-48d6-b478-5f30d9b1bb81 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.785737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 424/833] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:16.786963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3ed3-86fb-4ade-a45b-df7f3fc60bc5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.787250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3ed3-86fb-4ade-a45b-df7f3fc60bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.787524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3ed3-86fb-4ade-a45b-df7f3fc60bc5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.788860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 425/834] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:16.791308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c3ed3-86fb-4ade-a45b-df7f3fc60bc5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.818751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db6235-922d-4b8c-b51a-abe604462cc3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.826677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:16.855535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.856580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.857069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.857571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.869330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1c5ff581-01f2-4408-bfa8-e0bb97edbec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:16.875092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:16.881563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 410/835] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:16.882398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.883183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.883183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.896924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:16.898080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:16.916326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c3ed3-86fb-4ade-a45b-df7f3fc60bc5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:16.951156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:16.951738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:16.952014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:16.952287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/c1d8abb0b797488c8d07db05d6cc8892/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:16.971661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=c1d8abb0b797488c8d07db05d6cc8892, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:16.973107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:16.982450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 411/836] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:16.982450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:16.999974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 426/837] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:16 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/c1d8abb0b797488c8d07db05d6cc8892/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:28:17.028933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.096607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb8e94-69bc-4ce1-8e85-12af2d137bc8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.135217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.135852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.136162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.136434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.137113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:17.137410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.138978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.148663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 412/838] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:17.154506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.249993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.267915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.271313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e90c2-9f02-4cb8-8f53-f186f728c8c0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.315098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.315739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.316022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.316300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.330670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.332863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:17.352210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:17.353179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:17.354073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:17.354073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:17.354073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:17.354073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 427/839] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:17.359289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.386310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbf912-ea89-46bf-adcd-bf4d2f6b2ef3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.388532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.428821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.429882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.430371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.430840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.431572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.432948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:17.442199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:17.443211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:17.444121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 413/840] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:17.450825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.483510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce13d8-4e3d-4d8f-9d6b-efa179746431 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.519634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.520165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.520439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.520714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.521399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:17.521639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.523080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.524678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.539017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 428/841] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:17.544364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.585518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38529f-71e0-4c4b-a4e1-7154eb1583ec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.630305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.631414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.631929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.632442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.633362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:17.633939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.635754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.645990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 414/842] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:17.647232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.655075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f970e-6fa8-4767-af42-e46cd3ff820e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.680863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.692755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.693388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.693662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.693932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.702018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.703549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:17.716190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:17.717136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:17.718078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 429/843] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:17.723686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:17.768588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018680d4-bd3d-43d9-98bd-122ff16e09ff None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.800537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:17.810221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.811151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.811429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.811699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.812594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:17.812922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.814545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.826720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 415/844] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:17.857900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3545a5-d2c1-4599-a87f-ac64c5f47c34 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:17.896923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:17.897542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:17.897819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:17.898086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:17.898855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:17.899157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:17.900649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:17.920774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 430/845] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:17.927839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.040350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:18.047570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02476d1-bfcc-49f3-83fb-61f52962c812 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:18.085311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:18.085839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.086118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.086393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.158956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:18.160680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:18.258066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:18.268032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 416/846] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:18.272521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.379573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e0c7-a4ec-4029-a6a2-d9944cf66662 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:18.397996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:18.420534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:18.421168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.421444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.421714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.432725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:18.434171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-05a2199d-b84e-4715-823b-0dfc364808e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:18.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:18.450963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:18.452447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:18.452447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:18.452447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:18.452447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 431/847] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:18.457412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.499522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c669-cdb2-4890-8bfb-a7ea5dbe547a None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.499881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c669-cdb2-4890-8bfb-a7ea5dbe547a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.500151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c669-cdb2-4890-8bfb-a7ea5dbe547a None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.502329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 432/848] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:18.507268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.507539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.507813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.526724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:18.527940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:18.576384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20160b-8263-4d0b-9d35-43c6f6e4587f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:18.618720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 433/849] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:18.624786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:18.625419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.625696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.625966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.626602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:18.626928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:18.628337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:18.640596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 417/850] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:18.645677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.747791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:18.756791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcce383-369d-4364-8777-0ed4ab57958d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:18.792437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:18.793137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.793413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.793680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.800873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:18.802310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:18.814687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:18.815600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:18.816557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 434/851] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:18.821650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.881366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:18.891957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:18.935688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9db9-3678-47fa-a208-1e36cdfcf17b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:18.982074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:18.983342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:18.983855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:18.985335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:18.986536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:18.987339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:18.989402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.004080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957146-c0f0-4cbc-a9da-ca5b5a2e781e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.009085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 418/852] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:19.014606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.048102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.048738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.049041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.049289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.049881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:19.051292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-e8c3a488-7da8-4af9-afff-3f7ddedd83f3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:19.060207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:19.061162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:19.062047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 435/853] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 24 07:28:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:19.062919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51b0bf-50d4-42ec-9829-410c114afa26 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.062919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51b0bf-50d4-42ec-9829-410c114afa26 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.063903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51b0bf-50d4-42ec-9829-410c114afa26 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.068633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 436/854] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:19.068633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51b0bf-50d4-42ec-9829-410c114afa26 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.090674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:19.141743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dce972-f06c-405f-a00e-e36470168770 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.156486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:19.179568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.180230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.180456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.180731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.181449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=5c30642f91c54258b4ceec9005408be3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:19.181767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:19.181891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51b0bf-50d4-42ec-9829-410c114afa26 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.183286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64c431-28aa-4b75-94f5-a6d371bb4b36 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.189503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 419/855] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/role_assignments?user.id=5c30642f91c54258b4ceec9005408be3&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:19.190640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.190974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.191253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.211296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:19.212536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:19.239719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.241595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.242082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.242592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.243842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:19.244916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:19.247128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.255892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 437/856] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:19.260469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.303572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 420/857] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:19.361458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:19.381292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d1c57-1a5d-4c4b-afcc-e1170e0c5bee None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.420006 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.420944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.421430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.421908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.423354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:19.423879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:19.425513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.430756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 438/858] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:19.578192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.687456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11d205-57fa-4aa3-9f68-37c406a2a0cd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.697963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:19.726169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.726829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.727114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.727389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.738592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:19.743369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:19.744433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:19.745335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 421/859] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:19.752789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.856595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299510b-312c-43c9-a24e-d333821654f9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:19.894666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:19.895308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.895591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.895867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.896542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:19.896841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:19.898215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65a195-3c2e-4c42-b908-425ba8b40ae2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:19.908296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 439/860] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:19.913779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:19.921378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac334-c771-4109-8894-685fddeef1eb None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.921671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac334-c771-4109-8894-685fddeef1eb None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.921959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac334-c771-4109-8894-685fddeef1eb None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.924139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 440/861] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:19.929284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:19.929572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:19.929851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:19.952713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:19.954407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:19.956421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:20.030945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 441/862] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:20.033268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e649d-951b-4d1b-a7d9-b35d4411d066 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.068869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.069838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.070351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.070908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.081928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.083715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:20.098382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:20.099427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:20.100309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:20.100309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:20.100309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:20.102277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 422/863] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:20.111938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:20.221500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6dd5-4fc2-45f2-af7e-77c9cccebe7e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.257017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.257672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.257953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.258220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.258890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:20.259191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.260648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.268002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:20.273877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 442/864] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:20.279342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:20.301377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:20.303415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:20.407021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff7f72-4de9-458b-be62-1686936a4e3f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.413061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1cf71-ec33-415a-82c5-3310445bfba3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.442214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.442849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.443172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.443444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.444081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.445477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.458647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.459682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.460170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.460646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.467045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 443/865] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:20 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:28:20.470464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.472348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:20.486477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:20.487505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:20.488438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:20.491350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 423/866] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:20.496909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:20.569599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:20.624933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c1c9b-b4ec-4e0d-8d1e-97916ccf38b7 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.662174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.662809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.663087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.663443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.664164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:20.664466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.665883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.685168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 444/867] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:20.691798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:20.775144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:20.795187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da8611-8180-48ac-b1a0-60dc677c3b78 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:20.829250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:20.829900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:20.830178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:20.830445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:20.904264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:20.905736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:20.989472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:21.010634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 424/868] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:20 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:21.016810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.124966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166840-e238-4847-8f9b-4f55a4ba2060 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.140364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:21.163878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.165119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.165397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.165667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.177454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.178925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-b378b0c8-ee26-4a06-a223-fba509e4dadd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:21.194817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:21.195792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:21.196668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:21.196668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:21.196668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:21.196668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 445/869] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:21.202966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.295382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220f05f-e13f-4aba-b315-713f68729f6b None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.295705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220f05f-e13f-4aba-b315-713f68729f6b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.295933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220f05f-e13f-4aba-b315-713f68729f6b None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.297557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 446/870] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:21.302436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ed7-a7df-4d30-a1db-ce801b212994 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.302747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ed7-a7df-4d30-a1db-ce801b212994 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.303032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ed7-a7df-4d30-a1db-ce801b212994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.315157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643981f-fe96-46aa-a15b-70f47b81c677 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.322899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4f407ed7-a7df-4d30-a1db-ce801b212994 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:21.324212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4f407ed7-a7df-4d30-a1db-ce801b212994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:21.353239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.354106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.354391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.355126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.356056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:21.356365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.358401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.375666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 425/871] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:21.381080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.407821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 447/872] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:21.435066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:21.494710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f8659-f3ce-4ac3-8311-0a1485a0707f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.502127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0334-f256-4402-b749-6dc3c73e2a05 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.502410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0334-f256-4402-b749-6dc3c73e2a05 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.502704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0334-f256-4402-b749-6dc3c73e2a05 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.504553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 448/873] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:21.509365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.509645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.509909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.528145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:21.529323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:21.535630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.537024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.537511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.537985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.547683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.549873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:21.565998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:21.566945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:21.567925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:21.570681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 426/874] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:21.576243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.611403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 449/875] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:21.680760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.701365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:21.704335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b3c1-07da-49c4-9834-f5334ec64859 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.710420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:21.739894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.740584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.740928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.741262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.742020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:21.742389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.743937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.756528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 427/876] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:21.762334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.801559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d9e27-2da5-46b2-8d57-e6a5df112e54 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.843870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.844802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.845284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.845729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.846467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.848220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:21.857937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:21.858975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:21.860363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:21.867037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 450/877] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:21.867728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4ded4-88c8-414a-bc65-1d49e9f94c90 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:21.868734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.904139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:21.904836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:21.905177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:21.905511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:21.906291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=c1d8abb0b797488c8d07db05d6cc8892) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:21.907358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:21.908131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:21.914180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 428/878] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/role_assignments?user.id=c1d8abb0b797488c8d07db05d6cc8892&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:21.914896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:21.950536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:22.005140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:22.022317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e155f81-3c28-4353-9314-98c18373a4b2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:22.040198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d56a5-cc50-4661-adef-754cac4fe5d7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:22.073018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514879b7-39de-48af-b692-15d472653d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:22.073633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514879b7-39de-48af-b692-15d472653d74 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:22.073911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514879b7-39de-48af-b692-15d472653d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:22.074178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514879b7-39de-48af-b692-15d472653d74 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:22.074881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514879b7-39de-48af-b692-15d472653d74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:22.075232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514879b7-39de-48af-b692-15d472653d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:22.076560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514879b7-39de-48af-b692-15d472653d74 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:22.083559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 451/879] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:22.088923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514879b7-39de-48af-b692-15d472653d74 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:22.093625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:22.094276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:22.094555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:22.094899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:22.095561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:22.096963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dcf0d-ff5f-42a7-b611-0bfea22289a1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:22.119289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 429/880] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:21 2022] POST /identity/v3/services => generated 269 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:22.221103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514879b7-39de-48af-b692-15d472653d74 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:22.268976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:22.269585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:22.269859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:22.270125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:22.270928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:22.271223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:22.272577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9abe7-014d-4218-b186-50e9ef7016ff None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:22.278381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 452/881] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:22.561274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e03f7-5ac2-46a2-a08e-eb51e7308d4a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:22.561618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e03f7-5ac2-46a2-a08e-eb51e7308d4a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:22.561723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e03f7-5ac2-46a2-a08e-eb51e7308d4a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:22.563281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 430/882] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:22.569017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:22.569526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:22.570004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:22.600864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:22.602442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:22.611720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:22.677586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 453/883] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:22.954191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e03f7-5ac2-46a2-a08e-eb51e7308d4a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:23.045131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:23.064326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e03f7-5ac2-46a2-a08e-eb51e7308d4a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:23.099561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:23.100537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:23.101016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:23.101507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:23.102310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:23.103917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:23.114014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:23.115040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:23.115849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:23.120114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 431/884] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:23.126726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:23.235768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a80ebe-0f31-48c9-82cd-703f991851c4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:23.271314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:23.271928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:23.272209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:23.272491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:23.273131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:23.273436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:23.274838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:23.280087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 454/885] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:23.285516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:23.393202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945928c8-e251-449f-ba42-f68ed67b8734 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:23.427283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:23.428203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:23.428722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:23.429195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:23.430042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:23.430626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:23.432284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:23.439404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 432/886] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:23.445900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:23.456055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:23.555256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70037-b885-4c7a-8501-eb035a04be02 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:23.596920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:23.597538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:23.597816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:23.598196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:23.598949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] RBAC: Adding query filter params (interface=public, service_id=039439885d364f598b5eff1e3b39003c, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:23.599259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:23.600619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0549-476c-4981-a5b4-a6d0af7e09f0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:23.606291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 455/887] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:23 2022] GET /identity/v3/endpoints?service_id=039439885d364f598b5eff1e3b39003c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:23.611826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:23.639601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:23.720272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904106ba-8791-4882-9217-5eccf42656e4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:23.756076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:23.756691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:23.756971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:23.757246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:23.757917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:23.759330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2497-cbe4-4946-907b-ebb74771fc67 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:23.765561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 433/888] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:23 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:28:24.101616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8e10-21b6-4d05-b67c-642ce9e0ec12 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.102336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8e10-21b6-4d05-b67c-642ce9e0ec12 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.102861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec8e10-21b6-4d05-b67c-642ce9e0ec12 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.104705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 456/889] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:24.110678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af65f-28da-49d6-aa72-829b14411bf0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.111674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af65f-28da-49d6-aa72-829b14411bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.112158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af65f-28da-49d6-aa72-829b14411bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.132331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ee5af65f-28da-49d6-aa72-829b14411bf0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:24.133877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ee5af65f-28da-49d6-aa72-829b14411bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:24.163217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cdaf2-f4f2-46b5-99d9-285b36fb6393 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.163529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cdaf2-f4f2-46b5-99d9-285b36fb6393 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.163759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cdaf2-f4f2-46b5-99d9-285b36fb6393 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.165223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 457/890] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:24.170318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78aba0-fcf3-4070-a287-9c6fc60f7013 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.170600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78aba0-fcf3-4070-a287-9c6fc60f7013 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.170897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78aba0-fcf3-4070-a287-9c6fc60f7013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.197051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2e78aba0-fcf3-4070-a287-9c6fc60f7013 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:24.198261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2e78aba0-fcf3-4070-a287-9c6fc60f7013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:24.215075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 434/891] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:24.274098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:24.279046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 458/892] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:24.317597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c15d-eda3-4978-8d31-71354ac98a40 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.318244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c15d-eda3-4978-8d31-71354ac98a40 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.318756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c15d-eda3-4978-8d31-71354ac98a40 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.320594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 435/893] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:24.328466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.329292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.329566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.348499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:24.349673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:24.424106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 459/894] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:24.520599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54c15d-eda3-4978-8d31-71354ac98a40 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.532696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:24.564077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.647735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54c15d-eda3-4978-8d31-71354ac98a40 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:24.684258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:24.685197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.685992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.686680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.697128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e972f29-4c27-4ace-a745-e5dc222c7d5a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:24.699632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:24.704687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:24.705730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 436/895] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:24.711393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.738186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:24.745553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:24.746186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.746466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.746848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.747368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:24.748798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:24.757849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:24.758807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:24.759702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:24.764191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 460/896] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 24 07:28:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:24.764776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.782575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:24.820995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2c26c-a759-4d29-ac9c-11f61b1db7a6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:24.857964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:24.858592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.858914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.859191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.859891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:24.861326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:24.877043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 437/897] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:24 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:24.877788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4571c-2fce-47ad-831f-ab32fd593a4d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:24.877902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.930673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:24.931816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:24.932413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:24.932883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:24.933942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:24.934584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:24.936496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:24.949203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 461/898] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:24.955782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:24.996552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10407e-dbe1-4271-a42e-9ddf39189a16 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.022140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.024917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.040622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.041273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.041516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.041786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.042422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:25.042750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.044131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.049428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 438/899] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:25.055219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.089519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b78fe-37f8-495c-846c-c95e5f5e197b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.138746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.139406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.139681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.139953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.162053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.163533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:25.179553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:25.180695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:25.181630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:25.181630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:25.181630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:25.181630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 462/900] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:25.185211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576d4d3-6b3f-4ef1-81ae-de82f682956f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.187687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.231685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.232550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.232977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.233427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.234200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:25.235114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.236633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.243938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 439/901] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:25.253633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.304573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e724-1998-405e-9916-4687d08fe111 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.308153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.329676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.349385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.350166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.350446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.350749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.352193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:25.352511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.353984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.369315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 463/902] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:25.374364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.395081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a3e3-fbb9-4c75-b4a2-63da9124d80f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.429968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.430619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.430940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.431209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.431901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] RBAC: Adding query filter params (interface=public, service_id=d24327d8cc3241538ac3c341f1bbcfd3, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:25.432200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.433552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.446357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 440/903] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/endpoints?service_id=d24327d8cc3241538ac3c341f1bbcfd3&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:25.448112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.496627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708fb52-4905-461a-b092-86e75b269b6e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.535272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.536287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.536775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.537260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.545578 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.547475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.554410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df3a6-a43c-4c2e-8c79-177a022c1477 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:25.560959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:25.561882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:25.563306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:25.565872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 464/904] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:25.573931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.575331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.596378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.597101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.597433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.597702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.598369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.599864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf108-9ee9-4838-841f-3922a97a29a0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.606514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 441/905] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:28:25.615113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.697217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4008-ab9a-4ce8-a165-c5cd8523d325 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.732123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6725-2aa1-4560-89f3-4a9330112a11 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.732405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6725-2aa1-4560-89f3-4a9330112a11 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.732687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6725-2aa1-4560-89f3-4a9330112a11 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.734297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 442/906] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:25.735074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.735875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.736330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.736750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.737649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:25.738102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:25.739843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:25.740024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.740314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.740589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:25.760083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:25.761682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:25.762775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 465/907] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:25.772013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:25.859224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 443/908] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:25.884704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:25.904334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e935ae-47ce-4f0e-896f-dfd2425e4577 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:25.941276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:25.942310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:25.942828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:25.943337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.017283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.019088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.108475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.135191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.136088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 466/909] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:25 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:26.144304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.269053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8e2b-8f61-4e8f-911c-2de9a553f9a2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.269248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50054975-9c4a-4fb3-af95-11a2a646b989 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.284388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.307529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.308989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.309100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.309602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.309709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.309949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.310220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.310592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.310831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.312408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:26.322299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:26.323338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:28:26.324169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:26.325471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.327056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 444/910] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:26.327364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.335645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:26.342994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:26.344032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:26.345483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:26.345483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:26.345483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:26.349362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 467/911] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:26.350347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.385214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.465101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7f24-d5ba-4920-b241-16f73125ffa5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.479402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598646-d3ee-4ad5-8e03-7cda2c32c964 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.501237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.502013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.502606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.502763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.503497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:26.503896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.505359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.515615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 445/912] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:26.517153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.524086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c259094-f470-4678-b190-8845787892f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.524886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259094-f470-4678-b190-8845787892f6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.525309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259094-f470-4678-b190-8845787892f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.525736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259094-f470-4678-b190-8845787892f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.526587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c259094-f470-4678-b190-8845787892f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:26.527053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c259094-f470-4678-b190-8845787892f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.529053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c259094-f470-4678-b190-8845787892f6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.546613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 468/913] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:26.552176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c259094-f470-4678-b190-8845787892f6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.572399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.647702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2779-8b5a-44da-ae5c-ca7f390c6408 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.667460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.680228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c259094-f470-4678-b190-8845787892f6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.690908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.691827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.691827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.692085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.692665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:26.692962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.694312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.699789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 446/914] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:26.705323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.727708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.729173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.729918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.730390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.741807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.743280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:26.754661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:26.755612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:26.756525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 469/915] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:26.763920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.836877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceaa5d-6fd5-463f-93da-965af3ca51b9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.840852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:26.877720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef6866-ca6e-4d12-90f2-f1fe2fc4da73 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:26.884464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.885260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.886193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.886616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.887444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.889773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5336a-f0b1-4697-b218-fc1472ae2150 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.916695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 447/916] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:28:26.923639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:26.924266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.924548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.924821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.925530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:26.925836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:26.927339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c18303-83fa-4a95-8d15-2c221a394181 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:26.937918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de33-cc79-45dc-ab07-0259494dbb16 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.940244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de33-cc79-45dc-ab07-0259494dbb16 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.940788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de33-cc79-45dc-ab07-0259494dbb16 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.947202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 470/917] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:26.947876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 448/918] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:26.950332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:26.950618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:26.951693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7de33-cc79-45dc-ab07-0259494dbb16 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:26.952364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:26.973734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:26.975232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:26.979656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:27.066484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 471/919] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:27.097886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7de33-cc79-45dc-ab07-0259494dbb16 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:27.135712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:27.136359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.136651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.136921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.137766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] RBAC: Adding query filter params (role.id=ba35abf2a1f645509159d2cf8e089d0e, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=c1d8abb0b797488c8d07db05d6cc8892) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:27.138073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:27.139597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:27.145697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 449/920] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:26 2022] GET /identity/v3/role_assignments?user.id=c1d8abb0b797488c8d07db05d6cc8892&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=ba35abf2a1f645509159d2cf8e089d0e => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:27.350600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:27.456491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d5069-06ae-4bbb-b256-acdb403678f1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:27.493101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:27.494095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.494595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.495101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.495906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:27.497469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-020427e6-7523-469b-b67f-b2e301d53a70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:27.507291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:27.508276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:27.509086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:27.511838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 472/921] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:27.518412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:27.547470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:27.622038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c90-30ea-4438-bd44-d83674bdd5ab None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.622241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c90-30ea-4438-bd44-d83674bdd5ab None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.622499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c90-30ea-4438-bd44-d83674bdd5ab None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.624135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 473/922] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:27.624253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ea86b-3f81-4ff5-9a90-398f169f6894 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:27.629823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.630102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.630380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.639364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:27.651025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:27.652361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:27.667042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:27.667841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.668126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.668406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.669413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:27.669987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:27.671546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:27.678234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 450/923] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:27.684516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:27.734209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 474/924] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:27.791654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:27.795818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f0a69-3091-4bb9-a755-3db66ac2c063 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:27.831706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:27.832552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.833012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.833465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.834291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:27.834805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:27.836335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:27.841621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 451/925] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:27.848188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:27.951036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f65ac-2885-4886-bac8-9d89cf31c07d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:27.987119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:27.988027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:27.988485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:27.988934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:27.989852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] RBAC: Adding query filter params (interface=public, service_id=79720d2045fc462a842507b78d6c4cde, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:27.990336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:27.991909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:27.993719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:27.997911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 475/926] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:27 2022] GET /identity/v3/endpoints?service_id=79720d2045fc462a842507b78d6c4cde&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:28.003467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:28.019819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:28.132239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac7bf0-ff55-47b2-932c-0167e8c4be2e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:28.133513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a03c-69b7-4dcc-945f-534949f71bd4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:28.166617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:28.167648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.168186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.168659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.169471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:28.171050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:28.176631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:28.177268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.177547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.177820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.178406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:28.179904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:28.180359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 452/927] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:28 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:28.195498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:28.196524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:28:28.197471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:28.201301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 476/928] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 24 07:28:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:28.208408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:28.234216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:28.325840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e854-4d79-4d79-8aeb-a77bd4fe322a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:28.364282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:28.365298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.365778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.366273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.367138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:28.367643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:28.369252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:28.374726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 453/929] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:28.381252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:28.446803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:28.488346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ef58-1454-404e-82d7-8c06e3d34130 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:28.524545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:28.525504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.525988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.526483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.527352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:28.527858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:28.529419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dc539-6670-4682-a8f4-788066b0c9ad None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:28.535707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 477/930] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:28.548406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:28.635580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:28.655620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b4439-1eb4-44bc-9eb3-81e063e176e4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:28.691678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:28.692291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.692571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.692841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.693502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:28.694903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17586200-c203-49f9-b73e-14d4c0f86011 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:28.718921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 454/931] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:28.968544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a9c3b-a19e-46c9-8140-c2ad3f707847 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.968873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a9c3b-a19e-46c9-8140-c2ad3f707847 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.969205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a9c3b-a19e-46c9-8140-c2ad3f707847 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.971793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 478/932] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:28.977484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:28.977773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:28.978045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:28.996713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:28.997949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:29.068214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a9d-a504-437c-bc09-e3aa39a6db6d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.068511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a9d-a504-437c-bc09-e3aa39a6db6d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.068902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a9d-a504-437c-bc09-e3aa39a6db6d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.070433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 479/933] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:29.075017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.075301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.075575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.081699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 455/934] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:29.097943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:29.099153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:29.171632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 480/935] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:29.360740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.426740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:29.441356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.472015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bfbc3-bcda-45e1-a9b8-3cef40cd02f7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.508126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:29.509059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.509608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.510244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:29.510990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.511863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:29.513494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:29.517017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING keystone.server.flask.common [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 07:28:29.538248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.sql.core [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:28:29.538248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:28:29.538248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [parameters: {'id': 'e573135d8c854bc0822a2da364eb091b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:28:29.538248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e573135d8c854bc0822a2da364eb091b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e573135d8c854bc0822a2da364eb091b. Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:28:29.547303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 07:28:29.548137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application yield resource Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise self.value Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application session.commit() Aug 24 07:28:29.549623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:28:29.550496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:28:29.551431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:28:29.552208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [parameters: {'id': 'e573135d8c854bc0822a2da364eb091b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.553048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:29.555617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e573135d8c854bc0822a2da364eb091b. Aug 24 07:28:29.557794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.559032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fdc6d-2f1e-4d66-891d-a02163f34f97 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.561002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 456/936] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Aug 24 07:28:29 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:29.567717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.598663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72779f67-c79f-4220-a296-130857b78536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:29.599328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779f67-c79f-4220-a296-130857b78536 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.599607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779f67-c79f-4220-a296-130857b78536 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.599882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779f67-c79f-4220-a296-130857b78536 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.613572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72779f67-c79f-4220-a296-130857b78536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-72779f67-c79f-4220-a296-130857b78536 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:29.618956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:29.619885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:29.620715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:29.622499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 481/937] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:29.628767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72779f67-c79f-4220-a296-130857b78536 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.699506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af3836-4c7f-46b1-9861-7db3010b9425 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.722896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:29.744996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72779f67-c79f-4220-a296-130857b78536 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.748734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:29.749267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.749545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.749817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.759045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:29.761649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:29.766987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:29.779098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:29.780147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:29.786321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 457/938] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:29.789014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.800475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:29.801348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.802235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.802668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.804060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:29.804664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:29.806680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:29.823125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 482/939] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:29.826556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:29.916887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87c4f1-72af-44a2-be50-75ad9d55dc90 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.954705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ce15-0f53-4cb9-a6b5-2fab0accedd1 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:29.955770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:29.956523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:29.956928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:29.957293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:29.958062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:29.959050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:29.960157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:29.986712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 458/940] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:29.999829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.001042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.001295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.001569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.014095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.015630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:30.033155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:30.034237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:30.035393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:30.035393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:30.035393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:30.035393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 483/941] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:30.042079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.090269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.158650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31bc5-5779-4606-a14c-ee44d2bd8646 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.195947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.196479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.196752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.197019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.197658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:30.197954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.199361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ac46e-7e13-48be-84f8-ef7214881eef None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.214069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 459/942] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:30.217287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.252025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.275999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7550-ba3a-412a-8af6-1fbf071eb4c5 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.276527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7550-ba3a-412a-8af6-1fbf071eb4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.276915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7550-ba3a-412a-8af6-1fbf071eb4c5 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.278639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 460/943] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:30.284370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.284775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.285175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.305655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:30.307481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:30.328912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dcaec-ae5d-4610-928a-0c8114d63e65 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.367472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.368105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.368384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.368646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.380125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.382303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:30.394473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:30.395624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:30.396779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:30.397440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 461/944] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:30.400375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 484/945] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:30.401704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.431653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.509102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef92fb-8a0b-4b46-b11e-9baebb5da303 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.544761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.545467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.545814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.546152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.546976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:30.547344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.548922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.574214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 462/946] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:30.575574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.665121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.680527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.693870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1204e-f73b-46c9-b46a-641bfb1a7fe9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.732174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.732807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.733088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.733360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.744720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.746263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.759721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-0c11cde7-c1b2-44a2-b927-1174cdd14664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:30.762080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:30.763040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:30.763956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:30.763956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:30.763956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:30.763956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 485/947] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:30.765401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3275f1-b63b-4d32-94f2-756d227ded4d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.765401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3275f1-b63b-4d32-94f2-756d227ded4d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.765401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3275f1-b63b-4d32-94f2-756d227ded4d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.767280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 486/948] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:30.771131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3275f1-b63b-4d32-94f2-756d227ded4d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.782982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576dc8-f5b2-47f1-9e81-4cfa60a55bbe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.819275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.819897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.820173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.820443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.821020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.822546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f1f2af60-424b-413f-9432-3830247a1fff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:30.832069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:30.833309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:30.834325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:30.838725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 463/949] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:30.840098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.840409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.840689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.861275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:30.863008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:30.882947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3275f1-b63b-4d32-94f2-756d227ded4d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:30.917185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:30.917792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:30.918069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:30.918342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:30.919025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:30.919317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:30.920682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:30.934164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 487/950] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:30.935253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:30.938132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:30.950256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 464/951] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:30.951719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.042856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0856d-63dc-44ba-a3b7-7b8566064776 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.065149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.073566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a7814-df68-4928-a038-cd829ebc4b00 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.081903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.082769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.083210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.083651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.084436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:31.084913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.086448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.091676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 488/952] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:31.098473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.118005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.119008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.119511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.120051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.128210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.129923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:31.142796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:31.143764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:31.144654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:31.151541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 465/953] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:31.152726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.175265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.213015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c0363-00ca-4fa3-9eeb-2552719da8d7 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.251945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.252556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.252831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.253099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.253732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:31.254036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.255434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.262727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 489/954] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:31.262727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.268767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0baeea-9b6a-4fe4-9187-599d39fde101 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.306593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.307222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.307554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.307912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.308840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:31.309346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.311744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.325253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.329846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 466/955] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:31.330997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.379031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdee96-1792-4ec5-962f-360d61996ec7 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.408994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.415280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.415915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.416198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.416473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.417055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.418419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:31.427838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:31.428948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:31.429921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 490/956] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:31.430785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.450549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf3ef-97df-4506-9875-8a77eb0cc42b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.488194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.488803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.489075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.489342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.489994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.491379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.522723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 467/957] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:31.522723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.536706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ffb15-4d3a-48e3-9dbd-354019c1a723 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.599599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.601108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.601599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.602337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/c1d8abb0b797488c8d07db05d6cc8892/roles/ba35abf2a1f645509159d2cf8e089d0e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.605503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=c1d8abb0b797488c8d07db05d6cc8892, role_id=ba35abf2a1f645509159d2cf8e089d0e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.607297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.613325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.631549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 491/958] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:31 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/c1d8abb0b797488c8d07db05d6cc8892/roles/ba35abf2a1f645509159d2cf8e089d0e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:28:31.676647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24632be-04bb-4f85-bf1d-e0d79292d6e7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.717325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.718253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.718731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.719173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.720017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:31.720474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.722133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05573-ce70-44af-aff7-2410e4997b45 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.733056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 468/959] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:31.734809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:31.794334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:31.853938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b1725-0cd9-478e-a158-23bcba9518ac None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:31.891033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:31.892354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:31.892635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:31.892955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:31.893602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:31.893905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:31.895309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37841b8-7d20-481a-8a05-36bdf6100625 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:31.900852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 492/960] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:32.037246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a123-9581-4367-8900-63463185d26c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:32.037575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a123-9581-4367-8900-63463185d26c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:32.037790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a123-9581-4367-8900-63463185d26c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:32.039435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 469/961] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:32.044822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:32.045111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:32.045387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:32.064749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:32.065934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:32.138525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 493/962] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:32.409107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1a123-9581-4367-8900-63463185d26c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:32.511030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1a123-9581-4367-8900-63463185d26c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:32.530855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:32.545511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:32.546402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:32.547094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:32.547368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:32.548545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:32.549979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:32.568342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.sql.core [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:28:32.568342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:28:32.568342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [parameters: {'user_id': 'b77ac0086bde40da96f0161386271e3c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:28:32.568342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:28:32.575199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:28:32.576112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application yield resource Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise self.value Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application session.commit() Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:28:32.577679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:28:32.578803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:28:32.579720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:28:32.580580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b77ac0086bde40da96f0161386271e3c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:32.582659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 07:28:32.584612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:32.585896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 470/963] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 24 07:28:32 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:32.586399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:32.631841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:32.692825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481cd8-8b2b-4774-9bc8-0bb94e02ce56 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:32.729145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:32.729802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:32.730082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:32.730354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:32.738971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:32.740456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-831d0fb7-6545-4bd5-91b9-b955e6e57a11 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:32.754140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:32.755221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:32.756783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 494/964] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:32.764350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:32.873712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0b0df-c4fb-4943-81a4-88d90ad82e90 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:32.908674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:32.909277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:32.909557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:32.909828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:32.910522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:32.910861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:32.912614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb8cd-07f7-47da-875e-bb808bd7a3e9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:32.951183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 471/965] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 24 07:28:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:33.600060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048053-6e5f-4edd-a7d2-3d26e2e5edfa None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.600358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048053-6e5f-4edd-a7d2-3d26e2e5edfa None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.600625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048053-6e5f-4edd-a7d2-3d26e2e5edfa None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.600809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4e3c-7cb2-47f1-831c-b03e5f7e97df None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.601064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4e3c-7cb2-47f1-831c-b03e5f7e97df None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.601349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4e3c-7cb2-47f1-831c-b03e5f7e97df None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.602103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 495/966] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:33.602857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 472/967] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:33.607621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a87fc-1d39-4f8b-89bf-3df25a372b04 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.607793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b432-9145-41fb-8975-fa769ddb641d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.607878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b432-9145-41fb-8975-fa769ddb641d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.607958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a87fc-1d39-4f8b-89bf-3df25a372b04 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.608180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b432-9145-41fb-8975-fa769ddb641d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.608281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a87fc-1d39-4f8b-89bf-3df25a372b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.627556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1a30b432-9145-41fb-8975-fa769ddb641d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:33.627769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2e9a87fc-1d39-4f8b-89bf-3df25a372b04 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:33.628730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2e9a87fc-1d39-4f8b-89bf-3df25a372b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:33.629085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1a30b432-9145-41fb-8975-fa769ddb641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:33.666301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:33.699977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 496/968] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:33.710339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 473/969] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:33.933304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae84fa-a849-403d-9b1c-4e44b4589dfd None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.933623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae84fa-a849-403d-9b1c-4e44b4589dfd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.933863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae84fa-a849-403d-9b1c-4e44b4589dfd None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.935406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 497/970] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:33.941216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:33.941644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:33.941822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:33.959943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:33.961124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:33.967720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae84fa-a849-403d-9b1c-4e44b4589dfd None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.004242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.029549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 474/971] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:34.030592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.079525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae84fa-a849-403d-9b1c-4e44b4589dfd None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.113770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.114399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.114668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.114966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.115544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.116896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:34.125527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:34.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:34.127779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 498/972] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:34.132308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.139281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.143880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4749053-c64f-465d-93e0-7f2bace92218 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.181047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.181672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.181949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.182220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.194144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:34.198850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:34.199787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:34.200631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 475/973] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:34.205669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.234198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.239007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e796-1c11-4abd-bfd4-2f6655264223 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.279401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.280017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.280296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.280572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.281221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:34.281527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.283568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.288827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 499/974] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:34.294479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.312001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9e99-ebb9-4140-8996-636435211f55 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.345984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.346591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.346923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.347202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.347866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:34.348170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.349536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.357372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 476/975] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:34.358143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.382180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.394787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1a4f6-2bca-4f64-9558-94c23972e77d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.429007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.429613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.429894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.430170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.430842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:34.431148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.432520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.439457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 500/976] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:34.440874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.459336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc009a0a-a16b-4116-b432-5de88ee4d347 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.468466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.493222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.493823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.494108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.494386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.495186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.496557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.510733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 477/977] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:34 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:34.511513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.549760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b5199-4652-4d92-8a25-8eb6fecac136 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.599465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.600118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.600400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.600670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.612719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.614326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.629058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:34.630163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:34.631405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:34.632369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:34.632369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:34.632369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:34.632369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 501/978] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:34.637808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.659664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6ea0-c6a5-41f0-9be9-8708c344153f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.709323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.710001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.710226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.710500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.711247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] RBAC: Adding query filter params (interface=admin, service_id=79720d2045fc462a842507b78d6c4cde, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:34.711556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.712937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.718422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 478/979] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/endpoints?service_id=79720d2045fc462a842507b78d6c4cde&interface=admin®ion_id=RegionOne => generated 193 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:34.723016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.724388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.749800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7e58-4c8b-4641-889f-f2b754f6b9c4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.786478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.787122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.787396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.787664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.788304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:34.788602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.789955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.803297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 502/980] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:34.808505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.836306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513ed3-5172-453d-9719-e0f21c542ed4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.873050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.873659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.873951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.874199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.874821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.876234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.887175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 479/981] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:28:34.912722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016523b-5a30-4e3c-a992-7dd6b2def471 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:34.950256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:34.951052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:34.951334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:34.951608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:34.954418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:34.962207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:34.963641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:34.977117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:34.978162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:34.979833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 503/982] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:34.984385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:34.998258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:35.092846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87f0b-6c7e-4185-ba52-9d6f2c46d3a2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:35.126252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:35.126887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:35.127164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:35.127434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:35.128154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:35.128456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:35.129932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:35.148154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 480/983] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 24 07:28:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:35.153599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:35.272468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76eda99-cef0-4679-94bf-56af10a14807 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:35.307573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:35.308187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:35.308466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:35.308736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:35.370821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:35.379138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:35.380744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:35.484575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 504/984] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:35 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:35.486685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:35.553564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:35.595544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728e1a2-6d29-4fe4-b653-1899196bbcdc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:35.632109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:35.632791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:35.633069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:35.633336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:35.643146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:35.644571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:35.658786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:35.659740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:35.665455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:35.665455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:35.665455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:35.665455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 481/985] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:35.667260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:35.759531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:35.768485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5c5e-2db8-4cd9-9208-a797175102ec None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:35.801760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:35.802355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:35.802630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:35.802937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:35.803698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:35.803983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:35.805388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:35.817231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 505/986] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:35.822570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:35.925851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791279b-2cdb-4792-a6dc-249721c97923 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:35.946104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:35.962231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:35.962915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:35.963196 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:35.963467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:35.971431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:35.972899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:35.984582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:35.985946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:35.987379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 482/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:35.991525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:36.092335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea1da7-697e-413c-a156-7a70c82d5679 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:36.125463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:36.126094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.126371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.126640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.127356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:36.127661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:36.129139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b995d-8f54-4636-aa35-5d5b8e7166d3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:36.142018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 506/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:36.147753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:36.247681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58643-dfea-4f5c-81ca-2ed2f9802781 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:36.282038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:36.283298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.283581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.283852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.284578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] RBAC: Adding query filter params (role.id=ba35abf2a1f645509159d2cf8e089d0e, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=c1d8abb0b797488c8d07db05d6cc8892) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:36.284882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:36.286371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074e38c-02d1-4b5c-9796-01768289295d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:36.292693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 483/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:36 2022] GET /identity/v3/role_assignments?user.id=c1d8abb0b797488c8d07db05d6cc8892&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=ba35abf2a1f645509159d2cf8e089d0e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:36.449500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b58b9-7beb-4209-b6b2-677e0c76e1e3 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.449783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b58b9-7beb-4209-b6b2-677e0c76e1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.450059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b58b9-7beb-4209-b6b2-677e0c76e1e3 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.451695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 507/990] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:36.457525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603f9-cd8f-441b-a840-f38ce6c6db4c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.457823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603f9-cd8f-441b-a840-f38ce6c6db4c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.458098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603f9-cd8f-441b-a840-f38ce6c6db4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.477046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-475603f9-cd8f-441b-a840-f38ce6c6db4c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:36.478235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-475603f9-cd8f-441b-a840-f38ce6c6db4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:36.522118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:36.554735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 484/991] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:36.808114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2d5c-f709-443e-809d-4d647e7ed527 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.808597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2d5c-f709-443e-809d-4d647e7ed527 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.808597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2d5c-f709-443e-809d-4d647e7ed527 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.810038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 508/992] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:36.816038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.816395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.816738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.819267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2d5c-f709-443e-809d-4d647e7ed527 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:36.835126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:36.836186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:36.836299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:36.907742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 485/993] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:36.932823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2d5c-f709-443e-809d-4d647e7ed527 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:36.968314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:36.968931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:36.969230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:36.969514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:36.970199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:36.971585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:36.980396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:36.981367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:36.982257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:36.987051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 509/994] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:36.987886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.085659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a851ad5-699a-4675-91d8-2ff346ea4718 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.087908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.120625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.121224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.121495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.121762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.122380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:37.122675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.124050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.129423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 486/995] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:37.134798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.174889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.215703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.239596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139cc67-0eab-45d0-b8a6-f593d3210b7d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.274339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.274966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.275251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.275536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.276147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:37.276441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.277797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.283397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 510/996] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:37.288802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.293062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d469b83-1d1b-4c91-a3c9-454099a9d21b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.333772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.334404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.334680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.334976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.335556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.336965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:37.347009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:37.347993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:37.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 487/997] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:37.354270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.386592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.402454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ef4c-34ad-4a94-902d-695726abb92d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.440422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.441034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.441310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.441581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.442283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] RBAC: Adding query filter params (interface=public, service_id=bcbadade918e41b38830237a8bd021df, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:37.442588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.443981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.449125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 511/998] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/endpoints?service_id=bcbadade918e41b38830237a8bd021df&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:37.454441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.463199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.465751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c36a-5bd5-452a-90d5-d6537ce6502b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.503963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.504650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.504841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.505112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.505735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:37.506033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.507445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.512408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 488/999] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:37.517873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.565326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e762f8-ae67-4ecb-8e51-447547b8f306 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.613794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.614456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.614803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.615170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.616007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.618074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.634730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 512/1000] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:28:37.635746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.644195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5b565-08d2-4b64-ab80-28238030e7bd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.683603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.684226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.684501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.684769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.685410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:37.685708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.687125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3133b68-53b4-4107-830a-6d261b2a3447 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.698302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 489/1001] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 24 07:28:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:37.699667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:37.776093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:37.811928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d7d-629a-4a71-8844-5265d0f3cea7 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:37.848498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:37.849454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:37.849934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:37.850431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:37.851326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:37.852818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf99bdb-18e2-4bf0-912d-3df6527e47cc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:37.872290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 513/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:38.242892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5269de-ad71-4fce-8a45-c53e42b0d7e1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:38.243395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5269de-ad71-4fce-8a45-c53e42b0d7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:38.243395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5269de-ad71-4fce-8a45-c53e42b0d7e1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:38.244864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 490/1003] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:38.249914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:38.250146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:38.250420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:38.270200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:38.271437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:38.350740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 514/1004] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:38.616541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5269de-ad71-4fce-8a45-c53e42b0d7e1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:38.720821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5269de-ad71-4fce-8a45-c53e42b0d7e1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:38.758036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:38.758667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:38.758974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:38.759248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:38.759835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:38.761200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:38.770423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:38.771583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:38.772400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:38.773655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 491/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 24 07:28:38 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:38.777512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:38.834873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:38.882645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086913ed-c72d-4f4e-9fc9-ae52604030a2 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:38.917941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:38.918550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:38.918851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:38.919122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:38.919745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] RBAC: Adding query filter params (name=image) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:38.920042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:38.921400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c5959-ef76-4927-80d5-75cb3b883968 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:38.930684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 515/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:38.931832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:38.959252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:39.036119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e8a4c-15ef-4cef-92fb-1acabce40ed0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:39.070550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:39.071404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:39.071683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:39.071953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:39.072636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:39.072961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:39.074315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027f5e8-9478-437c-bf4f-a572fd8aa1a5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:39.086808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 492/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:39.568310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23074-bb54-4f7a-9faf-e1644e9ed854 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:39.568734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23074-bb54-4f7a-9faf-e1644e9ed854 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:39.568913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23074-bb54-4f7a-9faf-e1644e9ed854 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:39.570354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 516/1008] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:39.576364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782e4-fd7c-4881-82bf-5cdfa2be6c76 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:39.576808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782e4-fd7c-4881-82bf-5cdfa2be6c76 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:39.576996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782e4-fd7c-4881-82bf-5cdfa2be6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:39.598191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aa3782e4-fd7c-4881-82bf-5cdfa2be6c76 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:39.599399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aa3782e4-fd7c-4881-82bf-5cdfa2be6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:39.670728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 493/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:39.817429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e6f8-6380-4c2b-a72a-f035dbf6cbb4 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:39.817767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e6f8-6380-4c2b-a72a-f035dbf6cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:39.817986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e6f8-6380-4c2b-a72a-f035dbf6cbb4 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:39.819550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 517/1010] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:39.824602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:39.824975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:39.825336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:39.843696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:39.844897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:39.858265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:39.913500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 494/1011] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:39.944918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761e6f8-6380-4c2b-a72a-f035dbf6cbb4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.048028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761e6f8-6380-4c2b-a72a-f035dbf6cbb4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.068596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:40.084828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.085393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.085672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.085945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.086532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.087927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:40.097204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:40.098202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:28:40.099428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:40.103004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 518/1012] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:40.104246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.194348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.208471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c172-35f7-41db-b79f-e022d3d8b889 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.244198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.244805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.245083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.245412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.246042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:40.246394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.247796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.253315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 495/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:40.259062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.273663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:40.300869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a75dc-4d03-4e1b-9738-2975709a8411 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.338382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.338952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.339237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.339513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.340618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.342050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.343803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 519/1014] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:40.349781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.366085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ebd73-152c-4377-9d35-4a0936ebf800 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.401091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.401703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.402101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.402376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.403037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:40.403396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.404899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.406961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:40.415987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 496/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 24 07:28:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:40.416901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:40.457958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89d315-520f-4357-ba24-81faf03a2370 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.495310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.495920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.496198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.496516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.497159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.498130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:40.498590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a335b0c-569e-4641-8be4-7288d62e01a1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.523792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410cb60-ddd9-47fe-a4b9-1eb0b754ad0b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:40.526796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 520/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:40 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:28:40.570737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:40.571353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:40.571634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:40.571904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:40.572569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:40.573971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad57ebb-8e5f-4e18-95e0-6cbcf5125a11 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:40.603423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 497/1017] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:41.000066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487963e-c67f-4b22-a636-f57bc73b8220 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:41.000557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487963e-c67f-4b22-a636-f57bc73b8220 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:41.000557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487963e-c67f-4b22-a636-f57bc73b8220 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:41.002527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 521/1018] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:41.007582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac134-4e57-4cea-96b6-9d144cb283dd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:41.007867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac134-4e57-4cea-96b6-9d144cb283dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:41.008139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac134-4e57-4cea-96b6-9d144cb283dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:41.026545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0d2ac134-4e57-4cea-96b6-9d144cb283dd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:41.027744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0d2ac134-4e57-4cea-96b6-9d144cb283dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:41.100548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 498/1019] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:41.373913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487963e-c67f-4b22-a636-f57bc73b8220 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:41.484733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487963e-c67f-4b22-a636-f57bc73b8220 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:41.487133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:41.522708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:41.523473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:41.523867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:41.524234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:41.524974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:41.526431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0da98-0ce8-410b-8bc3-8e7b6d45f06b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:41.559173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 522/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:28:41 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:42.531878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d0f6-bb9c-4b38-b29e-8c9ab39bcdc1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:42.532463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d0f6-bb9c-4b38-b29e-8c9ab39bcdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:42.532545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d0f6-bb9c-4b38-b29e-8c9ab39bcdc1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:42.533930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 499/1021] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:42.539327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:42.539711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:42.539910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:42.557814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:42.559040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:42.622515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c8c5-bbc9-4eec-b0e9-1c864accac52 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:42.622993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c8c5-bbc9-4eec-b0e9-1c864accac52 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:42.623093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c8c5-bbc9-4eec-b0e9-1c864accac52 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:42.624656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 500/1022] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:42.631595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:42.631930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:42.632163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:42.644543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 523/1023] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:42.652429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:42.653619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:42.707884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:42.723969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 501/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:42.908835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:42.999612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.007178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.011219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992fdc4-bc8a-4a0a-b36c-440ec3f2efd0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.046112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.046781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.047037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.047305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.048992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.050418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.052007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 524/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:43.057554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.114345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd5d59-b234-434b-8ccb-c29328dc472d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.151130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.152125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.152625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.152972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.162238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ef524-44a7-4844-9063-89c9b3eb9af5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.164756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:43.169498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:43.170449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:43.171345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 502/1026] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:43.176381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.202817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.203435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.203743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.204548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.205190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.206617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.230495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 525/1027] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:28:43 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:28:43.245166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.306267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379bab7-de57-471a-8846-9002231d3753 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.343691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.344312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.345122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.345399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.346080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:43.346382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.347804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8bf15-4e35-40fb-9c0b-9178de6b9cd4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.362781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 503/1028] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:43.364532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.382921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.461418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b3b1-c6ce-4b3a-8fc3-3403c7d08ea3 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.461611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b3b1-c6ce-4b3a-8fc3-3403c7d08ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.461877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b3b1-c6ce-4b3a-8fc3-3403c7d08ea3 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.463242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 504/1029] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:43.468510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.468823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.469102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.476443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf97b2-1730-450f-9325-0da6f13101a9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.487824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:43.489104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:43.515158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.515794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.516077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.516347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.527864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.529309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:43.544515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:43.546113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:43.547788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:43.547788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:43.547788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:43.547788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 526/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:43.551349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.563034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 505/1031] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:43.624550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.661101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaf9c7-04af-4487-8ebf-09ea62ca4d59 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.696198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.696809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.697089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.697362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.698015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:43.698316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.699725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.711918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 527/1032] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:43.717560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.762214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.824978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045fc3-e337-4b8a-b3f6-f6c5e03aa8c9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.838108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.861424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.862173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.862451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.862892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.871088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.872546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:43.884795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:43.885728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:43.886676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 506/1033] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:43.891853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:43.953078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacfc46-e901-4e97-a152-1e6a90a1f214 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:43.983376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:43.992764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:43.993394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:43.993666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:43.993933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] PATH_INFO: `/v3/services/image` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:43.994513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:43.995911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:44.004774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:44.005805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:44.007314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 528/1034] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:44.008629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.011688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.012057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63bfc-6a8e-403b-b9d3-6179cfae5ad7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.047427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.048586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.048864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.049135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.049936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:44.050153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.051766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.069147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 507/1035] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:44.075115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.123455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e4e2-05e5-46b8-91b5-213767aaa933 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.161523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.162170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.162444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.162794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.163423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] RBAC: Adding query filter params (name=image) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:44.163770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.165226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.175239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 529/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:44.176018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.176570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa171-dab5-44ef-bcac-54961344676f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.212465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.213070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.213350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.213622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.231390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.242777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.283465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ad9c7-1ed9-49a8-b101-7b9966e6c227 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.289666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.291154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.318883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.319610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.319948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.320293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.320987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] RBAC: Adding query filter params (type=image) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:44.321347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.322824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.327961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 530/1037] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:44.333996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.404110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 508/1038] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:28:44.405744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.446867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e02e4-1b0e-4073-b042-1cbddd31966d None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.465778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.466542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.484949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.485580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.485855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.486122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.486845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] RBAC: Adding query filter params (interface=public, service_id=919c3d2b053b411890fcdd9e86a8a29f, region_id=RegionOne) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:44.487153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.488503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.498539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 531/1039] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/endpoints?service_id=919c3d2b053b411890fcdd9e86a8a29f&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:44.499461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.515587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd699-70e4-4983-a92d-cb9dbb596d81 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.550787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.551411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.551688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.551958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.562233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.563718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:44.578801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:44.580281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:44.581238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:44.581238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:44.581238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:44.585208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 509/1040] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:44.586080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.615693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654c48-85a5-4404-9c4d-952413a6405b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.651834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.652447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.652726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.653000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.653605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.654995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.662717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 532/1041] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:28:44.692739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5d6c-b0db-44b7-b99b-995db6d744e9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.704892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.732149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.732770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.733047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.733316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.733970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:44.734270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.736327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.754266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 510/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:44.755433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:44.806888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:44.862612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081b239-a953-45e8-93c4-f2a0d86a38fc None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:44.895555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:44.896184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:44.896460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:44.896735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:44.903684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:44.905124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:44.916849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:44.917764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:44.918651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 533/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:44.925127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:45.024836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50ed2-10fc-44d4-b828-98381d734662 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:45.058534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:45.059154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.059431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.059697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.060379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:45.060679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:45.062095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0994e5-ce3f-4eaf-a363-02d5c4ed9416 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:45.078728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 511/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:45.081155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:45.182827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da273ac1-a512-4783-8b76-717c2fb3160c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:45.190795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:45.217861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:45.218464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.218763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.219032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.219806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] RBAC: Adding query filter params (role.id=ba35abf2a1f645509159d2cf8e089d0e, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:45.220109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:45.221530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:45.221809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa326-9529-4db0-add8-80fe56104650 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.222303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa326-9529-4db0-add8-80fe56104650 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.222803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa326-9529-4db0-add8-80fe56104650 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.225469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 512/1045] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:45.235011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 534/1046] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:45 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=ba35abf2a1f645509159d2cf8e089d0e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:45.237019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.237303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.237575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.257434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:45.259106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:45.286288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:45.337231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 513/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:45.603453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:45.710252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2906ef-b45e-40e0-971f-08a75d6c5464 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:45.744045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:45.744766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.745049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.745323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.757752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:45.762810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:45.763817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:45.765329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 535/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:45.770527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:45.870387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57707622-dca6-48e2-9cfb-0d6d2d8501ea None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:45.903566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:45.904157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:45.904435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:45.904703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:45.905348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:45.905659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:45.907018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:45.914956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 514/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:45.920455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:45.952360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:46.024241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e310-a1db-4699-950f-687bf3fbc755 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:46.058138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:46.059156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.059638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.060119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] PATH_INFO: `/v3/users/bcd17bd43df14e578802997d0fa5e5f6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.074558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] RBAC: Authorizing `identity:get_user(user_id=bcd17bd43df14e578802997d0fa5e5f6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:46.076197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:46.078355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 536/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:28:45 2022] GET /identity/v3/users/bcd17bd43df14e578802997d0fa5e5f6 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:46.083914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:46.109114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:46.186572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b7a88-70c9-446e-9f6a-913d49769d6c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:46.221520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:46.222120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.222395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.222662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.290998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:46.292464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:46.316698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:46.392642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 515/1051] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:46.398637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:46.496041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:46.509697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9b0e-dc58-4ef2-89b8-f516a0c271f6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:46.544442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:46.545078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.545359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.545634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.546781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:46.548782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce99e-e6de-4f5a-826c-1586f6660a1a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:46.555271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 537/1052] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:28:46 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:46.556689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:46.625474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169e61-68c5-4ee5-a0c7-678a799fa398 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.625754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169e61-68c5-4ee5-a0c7-678a799fa398 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.626077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169e61-68c5-4ee5-a0c7-678a799fa398 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.627587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 538/1053] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:46.632462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b33621-1c0b-4765-9bbd-0f19f6c888db None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.632749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b33621-1c0b-4765-9bbd-0f19f6c888db None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.633024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b33621-1c0b-4765-9bbd-0f19f6c888db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.652324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-58b33621-1c0b-4765-9bbd-0f19f6c888db None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:46.653512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-58b33621-1c0b-4765-9bbd-0f19f6c888db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:46.670212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f66ded-a5dc-4426-854f-1679ec936a58 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:46.694480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:46.708252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:46.708863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:46.709138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:46.709437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:46.710120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4, user.id=bcd17bd43df14e578802997d0fa5e5f6) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:46.710423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:46.711921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:46.717911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 516/1054] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:46 2022] GET /identity/v3/role_assignments?user.id=bcd17bd43df14e578802997d0fa5e5f6&scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:46.740599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 539/1055] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:47.016033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.115217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0727564-9fac-4481-888b-ac46d6d18af5 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.115501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0727564-9fac-4481-888b-ac46d6d18af5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.115782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0727564-9fac-4481-888b-ac46d6d18af5 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.117135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 540/1056] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:47.120172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c0464-b4f8-4811-9664-8f13bb72fe50 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.122437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.122910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.123185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.141184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:47.142990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:47.156932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.157589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.157867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.158137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.158733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:47.160090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:47.169234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:47.170243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:47.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 517/1057] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:47.176468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.219345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 541/1058] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:47.235552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:47.287758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c641-110d-489f-9e23-98db239583f8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.322015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.322908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.323391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.323949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.324763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] RBAC: Adding query filter params (name=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:47.325249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:47.326824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:47.332878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 518/1059] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:47.338236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.356642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:47.445156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c36ef7-516c-46f9-ae04-93d88ddc379e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.479757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.480675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.481168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.481634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.482548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] RBAC: Adding query filter params (type=image) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:47.483687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.484389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:47.486395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:47.496395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 542/1060] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:47.497374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.588067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:47.593283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dff86-81c7-4e6f-bdd6-b96c333e0f27 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.608460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:47.609960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549d970-4c0f-4b97-bd2f-2aef319ad96a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.631780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.632836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.633118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.633669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.646333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.646956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.647233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.647395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:47.647509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.648131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:47.649488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:47.652162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:47.653073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:47.653906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 519/1061] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:47.659617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.682782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 543/1062] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:28:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:47.778964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16efd25-bea0-4649-a1bb-4165cd1dfe2b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.827806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.828415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.828694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.828962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.829657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:47.829967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:47.831356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:47.840186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 520/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:47.846548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:47.952796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9a12-2d06-4a48-9941-08c60acebd39 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:47.987754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:47.988393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:47.988669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:47.988939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:47.991751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.001669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.003162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:48.018493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:48.019645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:48.020539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:48.020539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:48.020539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:48.022758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 544/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:48.025534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.057635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.128183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52810b-47cb-444d-aaea-0a3ceee9a601 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:48.163331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:48.163938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.164217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.164491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.165130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:48.165431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.166930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.178717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 521/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:48.184351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.284187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e077d4c-a842-4a63-a6ce-08a280fd4709 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:48.318233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:48.318905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.319186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.319458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.327490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.328965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:48.340165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:48.341110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:48.342049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:48.343238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 545/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:48.347199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.439995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.449784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7b94-9bb3-4c88-94dd-8d1d98042c5e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:48.482932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:48.483535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.483813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.484082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.484802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:48.485106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.487281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28627ba-c77a-489f-bdc2-7581c2a61c4d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.506457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 522/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:48.511942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.561017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.620342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63d09-7fbe-4631-8f3c-6cd313083ef0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:48.635361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882260-3c0f-4fe6-ba68-3d6ad44f4846 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.635656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882260-3c0f-4fe6-ba68-3d6ad44f4846 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.636049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882260-3c0f-4fe6-ba68-3d6ad44f4846 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.637455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 523/1068] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:48.643015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.643820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.644106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.657762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:48.658549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.658865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.659139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.661925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:48.663123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:48.670326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.671864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:48.686423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:48.687533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:48.688957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:48.688957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:48.688957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:48.688957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 546/1069] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:48.694196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.739792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 524/1070] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:48.802840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.806094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922977a8-045f-42e4-99b6-ff397762fff4 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:48.841287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:48.842094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:48.842537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:48.843004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:48.843796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:48.844238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:48.846527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:48.859498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 547/1071] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:48.864868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:48.945987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:48.966634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2e810-f13a-4bbe-9e94-4c10b33b263a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.001131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.001762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.002042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.002313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.005155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.009811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.011409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:49.023257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:49.024184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:49.025075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 525/1072] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:49.029991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.113674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657529fc-4363-484a-b930-62eff73c8c8a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.132678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8a8f-349c-49dd-9747-996d9150322c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.149058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b394000-930e-4316-984c-e4280028bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.149688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394000-930e-4316-984c-e4280028bafe None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.149965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394000-930e-4316-984c-e4280028bafe None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.150234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394000-930e-4316-984c-e4280028bafe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.160435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b394000-930e-4316-984c-e4280028bafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-7b394000-930e-4316-984c-e4280028bafe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:49.165128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:49.166045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:49.166908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 548/1073] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:49.170129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.170759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.171043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.171323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.172014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:49.172321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b394000-930e-4316-984c-e4280028bafe None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.172417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.174285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.184472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:49.190097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 526/1074] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:49.195456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.201486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:49.286480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b394000-930e-4316-984c-e4280028bafe None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.301944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fd344-8b17-46da-a8b0-2e651ab258ad None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.328827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.330015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.330676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.331073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.331769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:49.332074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.333434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.337128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.337784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.338065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.338328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/ba35abf2a1f645509159d2cf8e089d0e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.340607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=cff82f8d12d842d1baab39bd73ff54cd, role_id=ba35abf2a1f645509159d2cf8e089d0e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.341927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 549/1075] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:49.343004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.348014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.358908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 527/1076] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:49 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/cff82f8d12d842d1baab39bd73ff54cd/roles/ba35abf2a1f645509159d2cf8e089d0e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:28:49.439135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:49.460812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84b32a-6f7d-41e7-a027-b759bb2a20fa None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.500650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.503859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.504139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.504406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] PATH_INFO: `/v3/users/bcd17bd43df14e578802997d0fa5e5f6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.505663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] RBAC: Authorizing `identity:get_user(user_id=bcd17bd43df14e578802997d0fa5e5f6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.507153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa378b-ef45-4fcf-bb6d-981fff4278ce None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.509032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 550/1077] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:28:49 2022] GET /identity/v3/users/bcd17bd43df14e578802997d0fa5e5f6 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:49.514526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.575252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:49.626738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a4130-5ceb-400e-9ea9-c8dffc608464 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.647947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32485d-bd1c-4fe4-9b61-5c8e441f434d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.648238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32485d-bd1c-4fe4-9b61-5c8e441f434d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.648517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32485d-bd1c-4fe4-9b61-5c8e441f434d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.650148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 551/1078] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:49.654970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.655246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.655515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.661722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.662351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.662624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.662926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.664119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.665564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.667448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 528/1079] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:28:49 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:49.673538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:49.674756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:49.675938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:49.746801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 552/1080] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:49.779513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afbd1-f220-4e0c-afa4-e57bf1954dcb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:49.806914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:49.816142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:49.816792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:49.817068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:49.817335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4/users/bcd17bd43df14e578802997d0fa5e5f6/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:49.819980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4, user_id=bcd17bd43df14e578802997d0fa5e5f6, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:49.821400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:49.835853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 529/1081] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:28:49 2022] PUT /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4/users/bcd17bd43df14e578802997d0fa5e5f6/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:28:50.015201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:50.098777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:50.119252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113105-0a2a-4c5c-9c46-09d3d54f57c0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:50.152201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:50.152805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:50.153082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:50.153350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:50.220453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:50.221906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:50.302167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:50.323160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 553/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:50.328795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:50.426727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0778c-fbde-4bbf-8210-d1ee823a1103 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:50.461724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:50.462359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:50.462633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:50.462926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:50.474230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:50.475691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-27591179-dba1-4f77-9eb5-0a66a326b356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:50.489417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:50.490362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:50.491603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:50.491603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:50.491603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:50.494766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 530/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:50.496403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:50.603571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273fbb6-7d18-4f2d-8e84-c399a8aa51d7 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:50.637172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:50.637772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:50.638047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:50.638315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:50.638975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:50.639274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:50.640622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedcc5c-12dc-427d-9ee9-26c3fa329606 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:50.658740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 554/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:51.295988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559eb2-ffd7-4675-9e3b-671442ff4269 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.296341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559eb2-ffd7-4675-9e3b-671442ff4269 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.296626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559eb2-ffd7-4675-9e3b-671442ff4269 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.298637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 531/1085] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:51.303836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364eb9-7aeb-46b4-a75e-0b88a61d7366 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.304411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364eb9-7aeb-46b4-a75e-0b88a61d7366 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.304411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364eb9-7aeb-46b4-a75e-0b88a61d7366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.323258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e8364eb9-7aeb-46b4-a75e-0b88a61d7366 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:51.324444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e8364eb9-7aeb-46b4-a75e-0b88a61d7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:51.357897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:51.391555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 555/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:51.665482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59559eb2-ffd7-4675-9e3b-671442ff4269 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:51.708703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:51.733129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925cca3-df77-47e4-9589-4fb05724eeb6 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.733522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925cca3-df77-47e4-9589-4fb05724eeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.733801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925cca3-df77-47e4-9589-4fb05724eeb6 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.735178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 556/1087] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:51.740057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.740338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.740612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.758524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:51.759801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:51.769349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59559eb2-ffd7-4675-9e3b-671442ff4269 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:51.803816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:51.804446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.804729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.805002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.815240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:51.819935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:51.820952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:51.821803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 532/1088] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 24 07:28:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:51.826893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:51.833834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 557/1089] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:28:51.921308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:51.924206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88fa1d-f2d6-4923-a5ba-284d35a63962 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:51.959961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:51.960553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:51.960828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:51.961098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:51.961749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:51.962044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:51.963412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:51.971816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 533/1090] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:28:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:51.977591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.070492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.073517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72021c3e-ae6f-48fd-ac00-c9dc5484643c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.096527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.106785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.107425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.107706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.107982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.118596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.120062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:52.134809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:52.135940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:52.137139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:52.137139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:52.137139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:52.137969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 558/1091] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:52.141749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.199214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283725f-79ac-4d41-8355-fa295be52225 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.233590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.233871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.234146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.240789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d2c8-228f-45ae-8abd-868564760a2e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.244967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:28:52.249583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:52.250578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:52.251940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 534/1092] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:52.257067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.276368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.276970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.277247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.277522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.278165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:52.278464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.279906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.280313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.292891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 559/1093] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:52.298965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.313231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.356214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae227251-895a-4d72-8ca1-ea85f694e248 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.388679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.389278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.389622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.389901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.390561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:52.390890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.392255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.396061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c529-d4bc-43b9-a096-9fa6b4cadb1e None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.400775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 535/1094] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:52.406434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.430644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.431290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.431565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.431830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.439172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.440851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:28:52.451985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:52.452899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:52.453869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 560/1095] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:28:52.459357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.493249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.508920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178af61-7576-4fce-8ad2-57f689eb4e09 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.529289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.542485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.543128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.543406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.543676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] PATH_INFO: `/v3/users/bcd17bd43df14e578802997d0fa5e5f6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.544784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] RBAC: Authorizing `identity:get_user(user_id=bcd17bd43df14e578802997d0fa5e5f6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.546200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.547793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 536/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/users/bcd17bd43df14e578802997d0fa5e5f6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:52.556018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.559834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4da6b-59b7-4843-8782-c17cc692edbe None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.594575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.595243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.595519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.595788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.596515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:52.596816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.598232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.616096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 561/1097] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:52.621277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.659081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c067-a777-4d4f-a1dc-b1e4f8bc90a1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.692869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.693490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.693739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.694012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.714903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.720058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d16d-b4f2-44b7-8e1b-293e6baae122 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:52.748154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.755344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:52.755949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:52.756229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:52.756506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:52.762909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.764365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.823585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:52.825025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:52.864114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 537/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:28:52.870139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.920239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.929396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 562/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:28:52.930380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:52.959037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:52.968119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faea03-443d-48c6-99c1-6cf0b0c510e2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.003787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.004093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.004366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.004631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.005725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.007161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.009031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 538/1100] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:53.014899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:53.027912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedccda-af8c-44f6-a085-b75142c6cd74 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.061845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.062464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.062774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.063046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.073403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.074934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:28:53.089850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:53.090880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:28:53.091777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:28:53.091777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:28:53.091777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:28:53.091777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 563/1101] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:53.097124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:53.115409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ce60b-e9b3-4e51-a785-b49160f3f727 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.134478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:53.148362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.148968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.149250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.149525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.150241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4, user.id=bcd17bd43df14e578802997d0fa5e5f6) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:53.150548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.152004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.158149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 539/1102] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:53 2022] GET /identity/v3/role_assignments?user.id=bcd17bd43df14e578802997d0fa5e5f6&scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:53.185977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:53.209350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea76261-d1ae-4462-9445-e1a65b1b1206 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.246031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.246651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.246976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.247245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.247903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:53.248203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.249615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.262276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 564/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:28:53.268178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:53.371775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df058cc1-7597-4af3-8450-701e13f6b010 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.404731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.405279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.405559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.405832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.412477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.413908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:28:53.425462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:28:53.426534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:28:53.427488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 540/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:28:53.432458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:53.531978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2abf-eb94-4d80-8e26-e7f433536518 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.566366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.566996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.567278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.567553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.568253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:53.568559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.569999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4964e7-83fb-421a-bf36-a3cdac6928b2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.586432 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 565/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:53.589233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:53.634925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:53.689142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb096-9179-4a85-99d3-3dbb8f4f4e74 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:53.722751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:53.723349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:53.723619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:53.723895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:53.724619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] RBAC: Adding query filter params (role.id=ba35abf2a1f645509159d2cf8e089d0e, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=cff82f8d12d842d1baab39bd73ff54cd) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:53.724927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:53.726354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e29c18-f0b3-46e0-864f-a5f81f77504c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:53.732072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 541/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:53 2022] GET /identity/v3/role_assignments?user.id=cff82f8d12d842d1baab39bd73ff54cd&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=ba35abf2a1f645509159d2cf8e089d0e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:55.082012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44126e26-b11b-4d76-80d6-f6af869a1717 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:55.082538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44126e26-b11b-4d76-80d6-f6af869a1717 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:55.082538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44126e26-b11b-4d76-80d6-f6af869a1717 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:55.083913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 566/1107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:28:55.088960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:55.089344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:55.089509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:55.107525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:55.108689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:55.171794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 542/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:55.440181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44126e26-b11b-4d76-80d6-f6af869a1717 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:55.443137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:55.540689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44126e26-b11b-4d76-80d6-f6af869a1717 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:55.572979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:55.573587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:55.573865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:55.574137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:55.575265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:55.576690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce17284-7ae8-4073-a3d8-ea11d8758765 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:55.578566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 567/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:28:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:55.584597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:55.624959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:55.682757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579a60e-2a03-4924-a5d8-f58418e98390 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:55.715554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:55.716087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:55.716367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:55.716638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:55.717267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:55.718688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83ad52-e891-4849-a3e9-ed3f03ec53c8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:55.738574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 543/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:28:55 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:28:57.656301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a48f-73f8-4c13-8922-055c80d7d8e3 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:57.656791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a48f-73f8-4c13-8922-055c80d7d8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:57.656881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a48f-73f8-4c13-8922-055c80d7d8e3 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:57.658720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 568/1111] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:28:57.663881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:57.664293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:57.664434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:57.682612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:28:57.683818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:28:57.747503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 544/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:28:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:28:58.016489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325a48f-73f8-4c13-8922-055c80d7d8e3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:58.085793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:58.117866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325a48f-73f8-4c13-8922-055c80d7d8e3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:58.150396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:58.151033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:58.151396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:58.151668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] PATH_INFO: `/v3/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:58.153043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] RBAC: Authorizing `identity:get_role(role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:58.154399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:58.155710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 569/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:28:58 2022] GET /identity/v3/roles/428b94d9bca848c1a1180b457c176272 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:58.161389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:58.185594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:58.258847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730f88d-9949-4f7a-9a9b-6b31ed905e11 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:58.291442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:58.292001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:58.292280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:58.292555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] PATH_INFO: `/v3/users/bc5060c8f00f42e4a2ea602b88e3d887` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:58.305928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] RBAC: Authorizing `identity:get_user(user_id=bc5060c8f00f42e4a2ea602b88e3d887)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:58.307391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:58.308965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 545/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:28:58 2022] GET /identity/v3/users/bc5060c8f00f42e4a2ea602b88e3d887 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:58.314615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:58.410320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e265f2-3ccf-4db1-a3ef-0b80e16686b9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:58.446008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:58.446595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:58.446906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:58.447179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:58.455291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:58.512050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:58.513482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:58.609324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 570/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:28:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:28:58.615663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:58.694734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:58.712188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362434df-d96b-40dd-95ef-501f827d9346 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:58.744392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:58.745013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:58.745285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:58.745557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:58.746649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:58.748106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f02a37-6765-482e-abf5-abb0d18f4d25 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:58.749716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 546/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:28:58 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:28:58.756383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:28:58.794629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:28:58.853331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc5ae-d9c0-4b25-af8a-bd25f7c1c78c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:28:58.886179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:28:58.886795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:28:58.887072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:28:58.887342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:28:58.888057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4, user.id=bc5060c8f00f42e4a2ea602b88e3d887) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:28:58.888360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:28:58.889766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d25227-043a-42fd-b549-4d269580c23e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:28:58.894979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 571/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:28:58 2022] GET /identity/v3/role_assignments?user.id=bc5060c8f00f42e4a2ea602b88e3d887&scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4&role.id=428b94d9bca848c1a1180b457c176272 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:00.761556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e27b-ce4b-4fd7-ba2c-6c4c5a913e62 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:00.762173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e27b-ce4b-4fd7-ba2c-6c4c5a913e62 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:00.762173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e27b-ce4b-4fd7-ba2c-6c4c5a913e62 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:00.763485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 547/1118] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:00.769083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:00.769529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:00.769632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:00.787142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:00.788310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:00.851998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 572/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:01.117093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112e27b-ce4b-4fd7-ba2c-6c4c5a913e62 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:01.214483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112e27b-ce4b-4fd7-ba2c-6c4c5a913e62 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:01.247852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:01.248460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:01.248738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:01.249008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] PATH_INFO: `/v3/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:01.250379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] RBAC: Authorizing `identity:get_role(role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:01.251794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:01.253077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 548/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:01 2022] GET /identity/v3/roles/428b94d9bca848c1a1180b457c176272 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:01.258797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:01.288642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:01.358482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7b29f-4ecd-4b6a-ab2d-1a7210ae994e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:01.401097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:01.401648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:01.401925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:01.402197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] PATH_INFO: `/v3/users/bc5060c8f00f42e4a2ea602b88e3d887` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:01.403321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] RBAC: Authorizing `identity:get_user(user_id=bc5060c8f00f42e4a2ea602b88e3d887)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:01.404746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:01.406302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 573/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:01 2022] GET /identity/v3/users/bc5060c8f00f42e4a2ea602b88e3d887 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:01.412612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:01.431954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:01.510038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bc043-3669-419e-a2e6-43f0d9278a72 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:01.542343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:01.542980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:01.543260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:01.543534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:01.544637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:01.546042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842329c-db5a-4867-a688-053faf9120b7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:01.547665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 549/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:01 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:01.554072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:01.648255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:01.651235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded2ed7-3791-4a6e-accd-46cea9a78747 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:01.684083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:01.684721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:01.684995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:01.685262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4/users/bc5060c8f00f42e4a2ea602b88e3d887/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:01.687342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4, user_id=bc5060c8f00f42e4a2ea602b88e3d887, role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:01.688758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341ffc-16d6-4487-b103-f3a603bf453d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:01.703473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 574/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:29:01 2022] PUT /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4/users/bc5060c8f00f42e4a2ea602b88e3d887/roles/428b94d9bca848c1a1180b457c176272 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:29:03.585113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d6c5-aa99-4fcd-8248-434d1d2ecb6e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:03.585507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d6c5-aa99-4fcd-8248-434d1d2ecb6e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:03.585748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d6c5-aa99-4fcd-8248-434d1d2ecb6e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:03.587139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 550/1124] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:03.592434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642988c6-8256-49f5-90cd-30279807f422 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:03.592819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642988c6-8256-49f5-90cd-30279807f422 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:03.592979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642988c6-8256-49f5-90cd-30279807f422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:03.611797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-642988c6-8256-49f5-90cd-30279807f422 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:03.612989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-642988c6-8256-49f5-90cd-30279807f422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:03.677927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 575/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:03.945580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d6c5-aa99-4fcd-8248-434d1d2ecb6e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:04.029261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:04.044626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d6c5-aa99-4fcd-8248-434d1d2ecb6e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:04.077897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:04.078517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:04.078825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:04.079103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] PATH_INFO: `/v3/roles/428b94d9bca848c1a1180b457c176272` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:04.080504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] RBAC: Authorizing `identity:get_role(role_id=428b94d9bca848c1a1180b457c176272)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:04.081854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:04.083195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 551/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:03 2022] GET /identity/v3/roles/428b94d9bca848c1a1180b457c176272 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:04.088856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642988c6-8256-49f5-90cd-30279807f422 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:04.178185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:04.188400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642988c6-8256-49f5-90cd-30279807f422 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:04.222080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:04.222636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:04.222947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:04.223221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] PATH_INFO: `/v3/users/bc5060c8f00f42e4a2ea602b88e3d887` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:04.224331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] RBAC: Authorizing `identity:get_user(user_id=bc5060c8f00f42e4a2ea602b88e3d887)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:04.225765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:04.227360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 576/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:04 2022] GET /identity/v3/users/bc5060c8f00f42e4a2ea602b88e3d887 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:04.232977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:04.328847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6b287-aeb1-428e-9f46-d8b7e830dc27 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:04.360951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:04.361537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:04.361815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:04.362088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:04.384855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:04.426503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:04.427956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:04.521665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 552/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:29:04.527407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:04.626808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a34088-a6ee-44db-b3e1-429892761793 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:04.660525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:04.661148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:04.661433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:04.661706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] PATH_INFO: `/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:04.662837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1e1717db8f4dd1a0a6c1add1172ae4)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:04.664263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af113ee-9ed0-482e-83c4-1adb45625c82 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:04.665872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 577/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:04 2022] GET /identity/v3/projects/4f1e1717db8f4dd1a0a6c1add1172ae4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:04.671935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:04.724842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:04.768386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80046-9dfc-4169-9b76-ccee76b21549 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:04.801558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:04.802093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:04.802374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:04.802650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:04.803412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] RBAC: Adding query filter params (role.id=428b94d9bca848c1a1180b457c176272, scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4, user.id=bc5060c8f00f42e4a2ea602b88e3d887) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:04.803748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:04.805174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a431b3d-f32b-4d3e-b2a6-986b32f5b267 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:04.810780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 553/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:29:04 2022] GET /identity/v3/role_assignments?user.id=bc5060c8f00f42e4a2ea602b88e3d887&scope.project.id=4f1e1717db8f4dd1a0a6c1add1172ae4&role.id=428b94d9bca848c1a1180b457c176272 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:06.709260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f428141-9ce3-4c0b-92ce-8ccdd4063382 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:06.709795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f428141-9ce3-4c0b-92ce-8ccdd4063382 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:06.709795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f428141-9ce3-4c0b-92ce-8ccdd4063382 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:06.711158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 578/1131] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:06.716469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:06.716887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:06.717024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:06.738131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:06.739344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:06.804755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 554/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:07.074005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f428141-9ce3-4c0b-92ce-8ccdd4063382 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:07.100454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:07.171201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f428141-9ce3-4c0b-92ce-8ccdd4063382 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:07.204110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:07.205273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:07.205549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:07.205813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:07.206932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:07.208341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7f955-7a0c-450b-8038-86de92d340c6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:07.209856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 579/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:07.215551 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:07.262483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:07.310908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c453f39-3f81-4b54-88f7-531cd8fc5b80 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:07.344294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:07.344832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:07.345112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:07.345412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:07.346045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:07.347499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42d2c-bf51-4db0-9ac8-8e95c31608e9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:07.372552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 555/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:29:07 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:29:09.282279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70adc42-02f5-44f7-8acf-f0eb16f4abe3 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:09.282826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70adc42-02f5-44f7-8acf-f0eb16f4abe3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:09.282934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70adc42-02f5-44f7-8acf-f0eb16f4abe3 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:09.284272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 580/1135] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:09.289583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:09.289972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:09.290137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:09.308546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:09.309738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:09.374457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 556/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:09.644879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70adc42-02f5-44f7-8acf-f0eb16f4abe3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:09.736511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:09.744401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70adc42-02f5-44f7-8acf-f0eb16f4abe3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:09.778431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:09.779070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:09.779349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:09.779618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:09.780714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:09.782139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96cb49-9edb-42b7-ba42-79c072c10727 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:09.783704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 581/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:09.789354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:09.818556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:09.886874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05696ccf-1d57-4a97-ade5-6ab2b94275b1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:09.918364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:09.918985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:09.919260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:09.919527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:09.920160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:09.921576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9d4d9-8fc7-409d-ad2e-e57f616a3552 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:09.942345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 557/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:29:09 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:29:11.844092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db79b-50b1-4ae7-90cd-a5da21f092e6 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:11.844606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db79b-50b1-4ae7-90cd-a5da21f092e6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:11.844606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db79b-50b1-4ae7-90cd-a5da21f092e6 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:11.846473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 582/1139] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:11.851957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:11.852382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:11.852514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:11.870677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:11.871896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:11.936328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 558/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:12.203957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713db79b-50b1-4ae7-90cd-a5da21f092e6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:12.303273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713db79b-50b1-4ae7-90cd-a5da21f092e6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:12.336349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:12.336968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:12.337249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:12.337523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:12.347489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:12.352140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:12.353280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:12.354509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 583/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:12.359119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:12.376357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:12.455990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ff17-2ee1-4ecd-b291-637725be9b3e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:12.488957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:12.489549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:12.489827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:12.490101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:12.490841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:12.491147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:12.492518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:12.500261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 559/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:12.505928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:12.529990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:12.603237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5c95-7c10-4afe-b7dd-3a324e8d7df8 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:12.636154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:12.636765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:12.637038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:12.637303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] PATH_INFO: `/v3/users/8ca8a91754394450a6bfea6d753b121a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:12.649656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] RBAC: Authorizing `identity:get_user(user_id=8ca8a91754394450a6bfea6d753b121a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:12.651107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:12.652679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 584/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:12 2022] GET /identity/v3/users/8ca8a91754394450a6bfea6d753b121a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:12.659145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:12.735390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:12.756432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6109a8d-1cf7-424a-a3dd-666239a5cbbc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:12.788427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:12.789018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:12.789298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:12.789572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:12.854106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:12.855643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:12.928539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:12.950101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 560/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:29:12.956526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:13.035553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:13.054604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521494f-6d8d-4e4d-87af-023150a2d189 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:13.087704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:13.088325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:13.088603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:13.088875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] PATH_INFO: `/v3/projects/d84860a91fd04357aae05a14d641aeba` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:13.089979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] RBAC: Authorizing `identity:get_project(project_id=d84860a91fd04357aae05a14d641aeba)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:13.091435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aca44f-ab6f-4975-bbeb-abe2a4f99e3f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:13.093033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 585/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:12 2022] GET /identity/v3/projects/d84860a91fd04357aae05a14d641aeba => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:13.099164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:13.194290 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db1f5-0de0-4778-a77d-18f25ad600ad None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:13.226652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:13.227261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:13.227536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:13.227805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:13.228509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=d84860a91fd04357aae05a14d641aeba, user.id=8ca8a91754394450a6bfea6d753b121a) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:13.228813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:13.230307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237f854-162a-4217-84b0-78da07f54fc4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:13.235502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 561/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:29:13 2022] GET /identity/v3/role_assignments?user.id=8ca8a91754394450a6bfea6d753b121a&scope.project.id=d84860a91fd04357aae05a14d641aeba&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:15.091149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d39d-8378-4533-a1aa-326982626b51 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:15.091695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d39d-8378-4533-a1aa-326982626b51 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:15.091695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d39d-8378-4533-a1aa-326982626b51 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:15.093050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 586/1147] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:15.099234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:15.099646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:15.099784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:15.117308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:15.118478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:15.152373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:15.183546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 562/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:15.447051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d39d-8378-4533-a1aa-326982626b51 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:15.543520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d39d-8378-4533-a1aa-326982626b51 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:15.575916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:15.576520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:15.576791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:15.577055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:15.587619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:15.592293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:15.593263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:15.594109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 587/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:15.599854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:15.695342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3a18a-f802-48a8-a9b5-431572f7841e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:15.727848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:15.728384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:15.728663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:15.728935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:15.729586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:15.729909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:15.731346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:15.739077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 563/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:15.744732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:15.756365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:15.844035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8452b-2d73-42bc-a90c-7dfb73f0d199 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:15.876350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:15.877503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:15.877853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:15.878120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] PATH_INFO: `/v3/users/8ca8a91754394450a6bfea6d753b121a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:15.879249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] RBAC: Authorizing `identity:get_user(user_id=8ca8a91754394450a6bfea6d753b121a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:15.880655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:15.882189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 588/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:15 2022] GET /identity/v3/users/8ca8a91754394450a6bfea6d753b121a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:15.887788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:15.922252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:15.987348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458eb0cf-2b54-44ab-b1f7-bb72711a8616 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:16.018973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:16.019587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:16.019863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:16.020132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] PATH_INFO: `/v3/projects/d84860a91fd04357aae05a14d641aeba` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:16.021215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] RBAC: Authorizing `identity:get_project(project_id=d84860a91fd04357aae05a14d641aeba)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:16.022623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884234-23ea-442b-b634-e552409d4174 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:16.024255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 564/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:15 2022] GET /identity/v3/projects/d84860a91fd04357aae05a14d641aeba => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:16.030205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:16.105367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:16.127839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ca3e1-e34c-486b-a134-be8a1491b72c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:16.161861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:16.162504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:16.162807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:16.163081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] PATH_INFO: `/v3/projects/d84860a91fd04357aae05a14d641aeba/users/8ca8a91754394450a6bfea6d753b121a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:16.164955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d84860a91fd04357aae05a14d641aeba, user_id=8ca8a91754394450a6bfea6d753b121a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:16.166375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b59ee4-7f5a-4107-a1bc-6265ee18dcb4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:16.179278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 589/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:29:16 2022] PUT /identity/v3/projects/d84860a91fd04357aae05a14d641aeba/users/8ca8a91754394450a6bfea6d753b121a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:29:18.036443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abaca-60b9-4fbe-b363-16833ec80f20 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.036972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abaca-60b9-4fbe-b363-16833ec80f20 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.036972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abaca-60b9-4fbe-b363-16833ec80f20 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:18.038382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 565/1154] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:18.043596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.044019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.044142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:18.062197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:18.063413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:18.127245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 590/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:18.394171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abaca-60b9-4fbe-b363-16833ec80f20 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:18.493075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abaca-60b9-4fbe-b363-16833ec80f20 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:18.494574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:18.527721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:18.528341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.528619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.528892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:18.538668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:18.543387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:18.544467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:29:18.545671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 566/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:18.550462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:18.613888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:18.650008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e5644-a302-4d91-b42b-18e15d85e2c5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:18.683525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:18.684120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.684399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.684673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:18.685341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:18.685646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:18.687035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:18.695663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 591/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:18.701593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:18.797918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a893a0-044e-4c5e-804c-3c7d07b5c4ec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:18.831567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:18.832184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.832463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.832738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] PATH_INFO: `/v3/users/8ca8a91754394450a6bfea6d753b121a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:18.833841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] RBAC: Authorizing `identity:get_user(user_id=8ca8a91754394450a6bfea6d753b121a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:18.835315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:18.836866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 567/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:18 2022] GET /identity/v3/users/8ca8a91754394450a6bfea6d753b121a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:18.842865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:18.938635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072383f4-cb03-459f-a066-470273a3e7f0 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:18.964338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:18.972867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:18.973557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:18.973868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:18.974180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:19.039781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:19.041226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:19.135430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 592/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:29:19.141483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:19.171775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:19.238867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3f586-664c-453b-87e2-c0c2052ff446 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:19.275447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:19.276056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:19.276329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:19.276598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] PATH_INFO: `/v3/projects/d84860a91fd04357aae05a14d641aeba` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:19.277684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] RBAC: Authorizing `identity:get_project(project_id=d84860a91fd04357aae05a14d641aeba)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:19.279121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2d90a-54a0-4aa2-bf78-a5511ddc008b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:19.280707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 568/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:19 2022] GET /identity/v3/projects/d84860a91fd04357aae05a14d641aeba => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:19.286768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:19.313042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:19.383631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8fc9c-6287-42d3-ac2c-5f27eff44640 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:19.416298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:19.416890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:19.417180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:19.417436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:19.418150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=d84860a91fd04357aae05a14d641aeba, user.id=8ca8a91754394450a6bfea6d753b121a) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:19.418454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:19.419897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950bc3-2d7d-4b6c-bbec-38e400ee7f36 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:19.425996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 593/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:29:19 2022] GET /identity/v3/role_assignments?user.id=8ca8a91754394450a6bfea6d753b121a&scope.project.id=d84860a91fd04357aae05a14d641aeba&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:21.326818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5e75-f49b-4b67-aec3-560c83f72c19 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:21.327341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5e75-f49b-4b67-aec3-560c83f72c19 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:21.327341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5e75-f49b-4b67-aec3-560c83f72c19 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:21.328684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 569/1162] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:21.334424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:21.334831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:21.335002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:21.353081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:21.354261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:21.417776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 594/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:21.689827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5e75-f49b-4b67-aec3-560c83f72c19 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:21.786484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:21.791387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5e75-f49b-4b67-aec3-560c83f72c19 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:21.824645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:21.825238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:21.825520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:21.825791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:21.890270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:21.891724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:21.982048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:21.987107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 570/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:29:21.993011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:22.016728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:22.090319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fae61-3ced-48ba-9ced-b00894b02ccf None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:22.122796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:22.123408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:22.123685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:22.123956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:22.130384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:22.131848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-7d8bc4e9-32c0-4d26-8a53-ac8915c988f4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:22.143486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:22.144523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:22.145672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 595/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:29:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:22.150447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:22.245416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e0944-b718-448e-af37-ff4da856ecfa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:22.278142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:22.278737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:22.279153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:22.279422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:22.280045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:22.280341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:22.281675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19e03-a170-49ac-a561-22c6be2d53a3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:22.289744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 571/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 24 07:29:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:24.171146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0d5d-8bdf-4858-9878-082cf2342594 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:24.171699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0d5d-8bdf-4858-9878-082cf2342594 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:24.171699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0d5d-8bdf-4858-9878-082cf2342594 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:24.173026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 596/1167] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:24.178733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7cc8-9528-4183-98a2-c97b1971f73c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:24.179125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7cc8-9528-4183-98a2-c97b1971f73c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:24.179279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7cc8-9528-4183-98a2-c97b1971f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:24.197514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fb3b7cc8-9528-4183-98a2-c97b1971f73c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:24.198679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fb3b7cc8-9528-4183-98a2-c97b1971f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:24.252000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:24.263119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 572/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:24.531662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac0d5d-8bdf-4858-9878-082cf2342594 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:24.614701 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:24.633253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac0d5d-8bdf-4858-9878-082cf2342594 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:24.666219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:24.666878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:24.667154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:24.667422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:24.668079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:24.669436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66428f1-008d-4566-a882-cc693edc0f00 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:24.685436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 597/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Aug 24 07:29:24 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:29:26.606014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e136396-ba12-486a-83d3-79b35471c6e8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:26.606514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e136396-ba12-486a-83d3-79b35471c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:26.606608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e136396-ba12-486a-83d3-79b35471c6e8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:26.607946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 573/1170] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:26.613061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:26.613474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:26.613615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:26.631442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:26.632628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:26.698034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 598/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:26.960961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e136396-ba12-486a-83d3-79b35471c6e8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:27.059423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e136396-ba12-486a-83d3-79b35471c6e8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:27.093609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:27.094220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:27.094494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:27.094789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] PATH_INFO: `/v3/domains/54650637d5ce432ab94ae0a04af912aa` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:27.105426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=54650637d5ce432ab94ae0a04af912aa)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:27.106882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f226da4-3c0b-4bb7-b046-077e2f9062bb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:27.108440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 574/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Aug 24 07:29:26 2022] GET /identity/v3/domains/54650637d5ce432ab94ae0a04af912aa => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:27.114870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:27.182941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:27.212775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f8c07-b739-4e2d-af2c-42d9f48a52d1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:27.245493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:27.246084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:27.246365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:27.246637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:27.247297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:27.248727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e33b5-51ac-4ce0-bd9a-0abe74c95e61 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:27.273856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 599/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:29:27 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:29:29.170941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421add1-abf3-4ffc-9f9a-6b69e995bafa None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:29.171469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421add1-abf3-4ffc-9f9a-6b69e995bafa None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:29.171469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421add1-abf3-4ffc-9f9a-6b69e995bafa None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:29.172830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 575/1174] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:29.178727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:29.179131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:29.179256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:29.197579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:29.198782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:29.262301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 600/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:29.537218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421add1-abf3-4ffc-9f9a-6b69e995bafa None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:29.593399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:29.640779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421add1-abf3-4ffc-9f9a-6b69e995bafa None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:29.672459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:29.673076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:29.673352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:29.673620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] PATH_INFO: `/v3/domains/54650637d5ce432ab94ae0a04af912aa` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:29.675247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=54650637d5ce432ab94ae0a04af912aa)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:29.676665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e55a7-c31b-417b-b2b1-609cd4165093 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:29.678212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 576/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Aug 24 07:29:29 2022] GET /identity/v3/domains/54650637d5ce432ab94ae0a04af912aa => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:29.683911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:29.738365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:29.780782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf62fda-483e-42d6-ab99-e922217080fd None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:29.813832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:29.814369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:29.814646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:29.815445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:29.816096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:29.817776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e2e4f-078a-48e9-8d47-b259ae830ac8 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:29.838931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 601/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:29:29 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:29:31.746773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd18f5-dc06-4e15-ac99-2487374baeb6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:31.747333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd18f5-dc06-4e15-ac99-2487374baeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:31.747333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd18f5-dc06-4e15-ac99-2487374baeb6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:31.748644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 577/1178] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:31.754357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:31.754830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:31.754931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:31.773146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:31.774325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:31.838455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 602/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:32.104801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd18f5-dc06-4e15-ac99-2487374baeb6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:32.203429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd18f5-dc06-4e15-ac99-2487374baeb6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:32.223720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:32.238197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:32.238846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:32.239125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:32.239396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:32.249203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:32.253947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:32.255118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:29:32.256045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 578/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:32.261068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:32.301347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:32.358638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414bfe5-377b-438b-8df4-b791d9929962 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:32.390463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:32.391076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:32.391352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:32.391622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:32.392274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:32.392571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:32.394634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:32.402518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 603/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:32.408226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:32.504938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e536cf-53cc-4759-abaa-c7966b822ca7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:32.537645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:32.538249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:32.538525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:32.538822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] PATH_INFO: `/v3/users/739d5e633cf240e6b1f68b36b77fb537` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:32.551206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] RBAC: Authorizing `identity:get_user(user_id=739d5e633cf240e6b1f68b36b77fb537)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:32.552628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:32.554179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 579/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:32 2022] GET /identity/v3/users/739d5e633cf240e6b1f68b36b77fb537 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:32.560044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:32.657254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ead56-f66f-4dc2-9596-b8abf824a542 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:32.658739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:32.690954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:32.691550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:32.691827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:32.692353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:32.757568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:32.759032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:32.860975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 604/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:29:32.866976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:32.905628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:32.962817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db5b85-3d10-40c0-9af7-726987c474bd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:32.994484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:32.995121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:32.995394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:32.995663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] PATH_INFO: `/v3/projects/8084f7c051a84d54ad393361e9603d9b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:32.996740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] RBAC: Authorizing `identity:get_project(project_id=8084f7c051a84d54ad393361e9603d9b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:32.998152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8c314-a6d5-4d01-ad24-6a623d21d1fe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:32.999753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 580/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:32 2022] GET /identity/v3/projects/8084f7c051a84d54ad393361e9603d9b => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:33.006340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:33.008567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:33.104897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441e6c-b0e0-4140-b644-1d9b1cb3ead9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:33.138797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:33.139390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:33.139670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:33.139944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:33.140666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=8084f7c051a84d54ad393361e9603d9b, user.id=739d5e633cf240e6b1f68b36b77fb537) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:33.140972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:33.142383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3dc50-cdc0-46c0-a666-977f4138ca5b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:33.147706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 605/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:29:33 2022] GET /identity/v3/role_assignments?user.id=739d5e633cf240e6b1f68b36b77fb537&scope.project.id=8084f7c051a84d54ad393361e9603d9b&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:35.002662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b7ec-54ce-4559-b007-8da776b8a9bb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.003207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b7ec-54ce-4559-b007-8da776b8a9bb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.003322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b7ec-54ce-4559-b007-8da776b8a9bb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.004612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 581/1186] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:35.009642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.010033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.010190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.028033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:35.029226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:35.086037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:35.093576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 606/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:35.359366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791b7ec-54ce-4559-b007-8da776b8a9bb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:35.457092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791b7ec-54ce-4559-b007-8da776b8a9bb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:35.464211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:35.490984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:35.491597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.491874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.492145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.501901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:35.507162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:35.508254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:29:35.509268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 582/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:35.514249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:35.613046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1e20c-a82d-4b7f-a971-9dbe6983a336 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:35.645855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:35.646452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.646747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.647016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.647677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:35.647975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:35.649328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:35.656940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 607/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:29:35.662847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:35.759492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7fd1-c422-4f7e-9446-c3251b040d5d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:35.792122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:35.792726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.793000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.793273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] PATH_INFO: `/v3/users/739d5e633cf240e6b1f68b36b77fb537` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.794847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] RBAC: Authorizing `identity:get_user(user_id=739d5e633cf240e6b1f68b36b77fb537)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:35.796563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:35.798098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 583/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:35 2022] GET /identity/v3/users/739d5e633cf240e6b1f68b36b77fb537 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:35.804242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:35.863293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:35.907866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e917e3d-7357-4c7b-a0ec-9af2ef9a5742 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:35.944196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:35.944816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:35.945093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:35.945363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] PATH_INFO: `/v3/projects/8084f7c051a84d54ad393361e9603d9b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:35.946460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8084f7c051a84d54ad393361e9603d9b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:35.947920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e33e4e-4abc-4f97-a9f5-888439a224e1 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:35.949539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 608/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:35 2022] GET /identity/v3/projects/8084f7c051a84d54ad393361e9603d9b => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:35.955637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:35.989694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:36.053370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8fc3-74f4-425b-9332-094f9b8a58f4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:36.086550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:36.087162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:36.087443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:36.087716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] PATH_INFO: `/v3/projects/8084f7c051a84d54ad393361e9603d9b/users/739d5e633cf240e6b1f68b36b77fb537/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:36.089572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8084f7c051a84d54ad393361e9603d9b, user_id=739d5e633cf240e6b1f68b36b77fb537, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:36.091028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b66edb-0b81-42ed-bb89-dcaa19bda2b1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:36.103308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 584/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:29:35 2022] PUT /identity/v3/projects/8084f7c051a84d54ad393361e9603d9b/users/739d5e633cf240e6b1f68b36b77fb537/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:29:37.968386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01fa8-fa00-4a20-9049-7fb77b950480 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:37.968939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01fa8-fa00-4a20-9049-7fb77b950480 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:37.968939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01fa8-fa00-4a20-9049-7fb77b950480 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:37.970264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 609/1193] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:29:37.976001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:37.976434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:37.976534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:37.994276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:29:37.995469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:29:38.059509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 585/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:29:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:29:38.329053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da01fa8-fa00-4a20-9049-7fb77b950480 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:38.429777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da01fa8-fa00-4a20-9049-7fb77b950480 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:38.453757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:38.463035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:38.463651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:38.463928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:38.464198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:38.475038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:29:38.479679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:29:38.480757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:29:38.481616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 610/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:29:38.486764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:38.510150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:38.585158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136ccc8-b281-43f7-b3aa-0f62472e1438 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:38.618535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:38.619161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:38.619441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:38.619708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:38.620360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:38.620659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:38.622016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:38.630293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 586/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:38.636030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:38.732630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbe7b3-f856-4911-af6f-7fb5aee38b31 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:38.764908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:38.765519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:38.765797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:38.766069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] PATH_INFO: `/v3/users/739d5e633cf240e6b1f68b36b77fb537` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:38.767212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] RBAC: Authorizing `identity:get_user(user_id=739d5e633cf240e6b1f68b36b77fb537)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:38.768629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:38.770166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 611/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 24 07:29:38 2022] GET /identity/v3/users/739d5e633cf240e6b1f68b36b77fb537 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:38.775897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:38.859934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:38.875422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432810ae-9673-4ae5-9f6a-f8bb7d4fe529 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:38.908349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:38.908944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:38.909252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:38.909524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:38.974499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:38.976077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:39.055612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:39.071562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 587/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:29:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:29:39.077614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:39.135616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:39.180294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afeb83-eadb-4d36-afae-275552b4c14f None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:39.214186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:39.214829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:39.215104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:39.215370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] PATH_INFO: `/v3/projects/8084f7c051a84d54ad393361e9603d9b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:39.216457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] RBAC: Authorizing `identity:get_project(project_id=8084f7c051a84d54ad393361e9603d9b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:39.217969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e29adf-676c-41e9-825f-5fdc8f2fe98f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:39.219607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 612/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 24 07:29:39 2022] GET /identity/v3/projects/8084f7c051a84d54ad393361e9603d9b => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:39.225743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:29:39.322685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f90db-596e-4d57-bbba-53f5e348e5f7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:29:39.357086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:29:39.357678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:29:39.357955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:29:39.358225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:29:39.358972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=8084f7c051a84d54ad393361e9603d9b, user.id=739d5e633cf240e6b1f68b36b77fb537) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:29:39.359281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:29:39.360688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e92dfc-dd90-411b-9bca-41bba941bce6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:29:39.366283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 588/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:29:39 2022] GET /identity/v3/role_assignments?user.id=739d5e633cf240e6b1f68b36b77fb537&scope.project.id=8084f7c051a84d54ad393361e9603d9b&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:29:49.209985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=39 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:29:49.363741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=75 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:35.620298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f6e4-8f31-4f73-9e9b-7a46a36a7ef6 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:35.620889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f6e4-8f31-4f73-9e9b-7a46a36a7ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:35.621151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f6e4-8f31-4f73-9e9b-7a46a36a7ef6 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:35.624456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 613/1201] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:35.630226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:35.631225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:35.632175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:35.656116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:35.658274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:35.763198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 589/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:36.070617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515f6e4-8f31-4f73-9e9b-7a46a36a7ef6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:36.220096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515f6e4-8f31-4f73-9e9b-7a46a36a7ef6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:36.266498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:36.267302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:36.268751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:36.268751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:36.282004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:36.283691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:36.307476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:36.308756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:36.309796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:36.309796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:36.309796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:36.309796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 614/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:36.315005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:36.485572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aab83-215d-4f8e-b259-8c79d44340de None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:36.531960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:36.542298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:36.543111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:36.543562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:36.543902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:36.544639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:36.545184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:36.546799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05644fd0-e49c-413f-8960-cf88659b1ac2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:36.567343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 590/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:36.569004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:36.646535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:36.720964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ac9c2-16cd-4cd7-b1a5-0d3c72b69dce None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:36.774678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:36.775791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:36.776636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:36.777129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:36.778200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:36.779957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba91bb-1d70-411b-9041-e1c38f5412b5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:36.823242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 615/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 24 07:30:36 2022] POST /identity/v3/users => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:30:39.116487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedcd7-fd14-42ac-8554-c0fd28f63e11 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:39.117034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedcd7-fd14-42ac-8554-c0fd28f63e11 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:39.117034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedcd7-fd14-42ac-8554-c0fd28f63e11 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:39.118360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 591/1206] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:39.123814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601df363-70cb-4769-9157-a707769d000e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:39.124264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601df363-70cb-4769-9157-a707769d000e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:39.124433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601df363-70cb-4769-9157-a707769d000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:39.143533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-601df363-70cb-4769-9157-a707769d000e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:39.144759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-601df363-70cb-4769-9157-a707769d000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:39.224916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 616/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:39.494848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bedcd7-fd14-42ac-8554-c0fd28f63e11 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:39.611125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bedcd7-fd14-42ac-8554-c0fd28f63e11 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:39.647420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:39.648036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:39.648313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:39.648584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:39.659397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:30:39.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:39.665900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:39.666819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 592/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:39.672117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601df363-70cb-4769-9157-a707769d000e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:39.715879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:39.784833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601df363-70cb-4769-9157-a707769d000e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:39.820637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:39.821273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:39.821554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:39.822083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:39.822803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:39.823107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:39.824474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:39.833409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 617/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:30:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:30:39.839900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:39.893237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:39.956814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f982788-1a91-4831-a22a-a4ccfeb0bd4e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:40.015978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:40.015978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:40.015978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:40.015978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:40.027790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:40.029427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:40.046172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:40.047236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:40.053204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:40.053204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:40.053204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:40.053204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 593/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:40.053481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:40.187157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:40.189482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db6d2c-5bda-4315-9783-ed87bc093a62 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:40.240610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:40.241932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:40.242246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:40.242524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:40.243308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:40.243795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:40.245409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:40.261310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 618/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:40.267062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:40.396281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0787e-d5ef-480a-a16d-84c357937de2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:40.398220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:40.433742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:40.434444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:40.434744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:40.435020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:40.442679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:40.444701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:30:40.456256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:40.457476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:40.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 594/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:40.464053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:40.589318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ec16-4268-4632-95bd-f6a3438655d2 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:40.632303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:40.632946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:40.633190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:40.633497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:40.634192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:40.634508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:40.635962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:40.660455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 619/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:30:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:40.666230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:40.781092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522072-4ef4-44b4-aa05-5db095c8aedd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:40.821125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:40.821989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:40.822271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:40.822606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:40.857654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:40.900578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:40.902041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.021388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 595/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:30:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:30:41.027116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:41.048603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:41.146688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90497c0e-aecb-49b8-a1f3-72761e26626c None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:41.187662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:41.188531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:41.188617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:41.188878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:41.201607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:41.203103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:41.227421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:41.229165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:41.230674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:41.230674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:41.230674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:41.230674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 620/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:41.234729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:41.316596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:41.365018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff44124-fcdc-49df-98dc-c28de9654f31 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:41.404311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:41.404939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:41.405221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:41.405491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:41.406174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:41.406442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:41.407869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.422766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 596/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:41.426573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:41.507138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:41.542085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6c8c4-e492-4151-9c64-a186fa9178a6 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:41.586845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:41.587527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:41.587864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:41.588191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:41.597157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:41.599673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:30:41.614000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:41.615071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:41.616544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 621/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:41.622137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:41.731729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddc67c-77c4-41fd-b294-a375a2c0bc7f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:41.760461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:41.774002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:41.774613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:41.774923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:41.775197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:41.776073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:41.776379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:41.777859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840e71-8e3d-40b6-a1e6-18455b645e45 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.791007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 597/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:41.798234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:41.910353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a3a2d-809f-43d9-93c5-ae22cc0ef577 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:41.946223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:41.954346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:41.955192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:41.955772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:41.956224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:41.957418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=eef6061176284f0d912e496511a3c27a) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:41.957966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:41.960065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4622408-a634-43ee-97a4-282835bc8193 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:41.969455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 622/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:30:41 2022] GET /identity/v3/role_assignments?user.id=eef6061176284f0d912e496511a3c27a&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:43.843699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f17ad-97e8-4729-8ec3-73b0d596e0f6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:43.844192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f17ad-97e8-4729-8ec3-73b0d596e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:43.844192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f17ad-97e8-4729-8ec3-73b0d596e0f6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:43.845537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 598/1220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:43.851462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:43.851891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:43.852012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:43.872713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:43.873920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:43.951098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 623/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:44.239424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f17ad-97e8-4729-8ec3-73b0d596e0f6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:44.349525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f17ad-97e8-4729-8ec3-73b0d596e0f6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:44.386256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:44.386908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:44.387248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:44.387466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:44.398540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:30:44.403292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:44.404409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:44.405686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 599/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:44.410398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:44.516830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05b45c-ae76-4a2b-bf6e-75be4168064c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:44.541952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:44.552929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:44.555049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:44.555049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:44.555049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:44.555275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:44.555528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:44.556902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:44.567004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 624/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:44.573576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:44.620098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:44.677134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8931-c312-4381-87ff-91205aa06ddb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:44.713218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:44.713862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:44.714138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:44.714409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:44.727079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:44.728785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:44.743760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:44.749454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:44.750563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:44.750563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:44.750563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:44.750563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 600/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:30:44.750876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:44.852488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef238-ce04-47e4-94e6-81a4f94dd3d5 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:44.886610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:44.887234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:44.887511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:44.887787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:44.888425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:44.888724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:44.890089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:44.903949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 625/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:44.909804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.010772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:45.018479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140e4a0-f3e7-4717-a336-796bf67c19d4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.054963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.055588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.055867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.056146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.063690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.065177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:30:45.077178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:45.078271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:45.086375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 601/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:30:45.087155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.139346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:45.195697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b17c3-587a-49b0-9d1f-ab23a8765e21 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.231447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.232089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.232342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.232613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.233424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:45.233750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.235314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.252971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 626/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:30:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:45.258773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.360996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353a89-b21b-4fd5-bdfc-c5f5ef5183f3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.394410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.394997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.395280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.395551 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.407676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.409112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:45.423760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:45.424916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:45.426034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:45.426034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:45.426034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:45.426034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 602/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:45.432437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.538635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99347989-4360-4ed3-8ac5-b550d70def78 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.541251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:45.575835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.576436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.576747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.576986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.577621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:45.577980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.583875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.596474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 627/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:45.602441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.612409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:45.716217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36058b59-78a0-474a-bd06-a8b1b34e3fd9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.755948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.756572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.757390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.757666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.764271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.765708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:30:45.777811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:45.779095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:45.780362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:45.783859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 603/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:30:45.785253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:45.893512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308483-7952-454e-a984-4e469a7adb22 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:45.933047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:45.933652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:45.933931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:45.934234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:45.934924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:45.935249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:45.936713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044db60-53c6-4cbf-9e7e-8b190eecb9b0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:45.949618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 628/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:45.955657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:46.031578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:46.072050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425d62d-48f6-4c23-9279-20b4cb907ac3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:46.106486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:46.107174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:46.107449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:46.107721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/eef6061176284f0d912e496511a3c27a/roles/1908db315ff344ccb54df4c3bbe20f9d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:46.125648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=eef6061176284f0d912e496511a3c27a, role_id=1908db315ff344ccb54df4c3bbe20f9d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:46.127490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff521-bd5a-465e-978d-02f0cfaca877 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:46.141296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 604/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:30:45 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/eef6061176284f0d912e496511a3c27a/roles/1908db315ff344ccb54df4c3bbe20f9d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:30:48.112592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57589e-af0b-4ac2-9e7b-896cb8fbd9c1 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:48.113145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57589e-af0b-4ac2-9e7b-896cb8fbd9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:48.113145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57589e-af0b-4ac2-9e7b-896cb8fbd9c1 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:48.114633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 629/1233] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:48.120316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9571bc5-7586-484f-9704-255511269a3d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:48.120887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9571bc5-7586-484f-9704-255511269a3d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:48.120887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9571bc5-7586-484f-9704-255511269a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:48.141854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f9571bc5-7586-484f-9704-255511269a3d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:48.143584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f9571bc5-7586-484f-9704-255511269a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:48.217003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 605/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:48.485445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57589e-af0b-4ac2-9e7b-896cb8fbd9c1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:48.511157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:48.596350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57589e-af0b-4ac2-9e7b-896cb8fbd9c1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:48.631658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:48.632906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:48.633217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:48.633509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:48.644971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:30:48.650177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:48.651363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:30:48.652451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:48.656219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 630/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 24 07:30:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:48.657624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9571bc5-7586-484f-9704-255511269a3d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:48.706743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:48.760725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9571bc5-7586-484f-9704-255511269a3d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:48.794600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:48.795232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:48.795515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:48.795787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:48.796445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:48.796745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:48.798193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:48.806215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 606/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 24 07:30:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:48.812008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:48.906246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:48.923187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae597c4-0d73-4e41-81f9-8132b3a2aa7a None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:48.962990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:48.963729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:48.964051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:48.964363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:48.975162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:48.976668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:48.993363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:48.994560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:48.996019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:48.996019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:48.996019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:48.996019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 631/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:49.000727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:49.106886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94394e-b876-4587-a928-d5f5b87ce591 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:49.116024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:49.144921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:49.145610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:49.145789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:49.146065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:49.146724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:49.147023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:49.148405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:49.160787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 607/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:49.166549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:49.270850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc5978-de1f-4231-b720-449d6366cd04 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:49.304582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:49.305208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:49.305486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:49.305759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:49.308441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:49.313851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:49.315658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:30:49.327182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:49.328286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:49.329437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:49.334543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 632/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:49.334668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:49.438441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1075fc-cb99-47fe-9996-1bd42f9eed86 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:49.477010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:49.477687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:49.477999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:49.478316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:49.479312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:49.479650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:49.481319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:49.502883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 608/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:30:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:49.509458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:49.625248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6501e-b576-44cb-8886-0a1ac746c168 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:49.660245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:49.660880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:49.661158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:49.661433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:49.729758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:49.737977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:49.739545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:49.843802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 633/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:30:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:30:49.849956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:49.879942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:49.960142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3b34-069a-43b5-9a65-51b6e2ef3c0f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:49.999167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:49.999963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:50.000337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:50.000783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:50.013477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:50.015673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:50.034532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:50.035833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:50.036989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:50.036989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:50.036989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:50.036989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 609/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:50.042285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:50.151497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:50.157730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc8f4a-a197-4423-9b83-c5d8568435d3 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:50.192641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:50.193367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:50.193501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:50.193770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:50.194406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:50.194752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:50.196116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:50.207888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 634/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:50.213451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:50.297068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:50.322988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d156-738d-4da2-8231-f132ae0951fa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:50.360429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:50.361090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:50.361412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:50.361729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:50.368521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:50.369975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:30:50.382912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:50.384130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:50.385197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 610/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:50.390644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:50.496923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4b02-cdc2-414d-91a6-8304cfac6957 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:50.532907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:50.533705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:50.534022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:50.534363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:50.535181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:50.535557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:50.537169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589617f-a2b1-406f-88ac-8e1aef54bd3f None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:50.551096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 635/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:50.557155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:50.668775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72082572-58b2-4fb3-a9fd-efab5d786455 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:50.705121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:50.705813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:50.706097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:50.706366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:50.707189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] RBAC: Adding query filter params (role.id=1908db315ff344ccb54df4c3bbe20f9d, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=eef6061176284f0d912e496511a3c27a) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:50.707540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:50.709070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e28b0-a711-4438-b25d-fb957592839e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:50.715773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 611/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:30:50 2022] GET /identity/v3/role_assignments?user.id=eef6061176284f0d912e496511a3c27a&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=1908db315ff344ccb54df4c3bbe20f9d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:52.667304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a9c6-a620-488c-8272-3e139181ffcf None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:52.667847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a9c6-a620-488c-8272-3e139181ffcf None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:52.667847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a9c6-a620-488c-8272-3e139181ffcf None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:52.669164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 636/1247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:52.674406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:52.675149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:52.675149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:52.681195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:52.694071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:52.695324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:52.767503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 612/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:53.063145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9a9c6-a620-488c-8272-3e139181ffcf None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:53.084272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:53.183835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9a9c6-a620-488c-8272-3e139181ffcf None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:53.225864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:53.226634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:53.227054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:53.227328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:53.239621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:30:53.244734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:53.245734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:30:53.246574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:53.250896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 637/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:53.252381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:53.360626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4837c-1eb5-44a8-956c-0087b36faf00 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:53.400357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:53.408878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:53.409868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:53.410340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:53.410841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:53.412016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:53.412479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:53.414365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:53.426589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 613/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:53.434322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:53.522283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:53.548124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fef8c-2a84-4d54-a116-5672181a85bd None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:53.585684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:53.586327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:53.586604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:53.586972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:53.597412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:53.598913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:53.613716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:53.614760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:53.615672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:53.615672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:53.615672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:53.615672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 638/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:53.621297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:53.730026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186ee5-4953-46bc-b2de-8399bf54e8f5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:53.766791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:53.767472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:53.767755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:53.768024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:53.768666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:53.768967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:53.770332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:53.782454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 614/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:53.788233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:53.896917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3a653-0256-4840-b0b0-e1ecaa24ce42 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:53.926231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:53.935772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:53.936498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:53.936819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:53.937136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:53.945213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:53.946914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:30:53.959481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:53.960780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:53.961726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 639/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:53.966475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:54.029078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:54.085988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e47-b616-4c31-932f-52ce56102cc5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:54.125427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:54.126032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:54.126306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:54.126575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:54.127343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:54.127649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:54.129098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:54.148508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 615/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:30:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:54.154183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:54.264092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e8448-eca6-41e9-8f7a-37e19adf97bd None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:54.299814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:54.300415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:54.300694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:54.300962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:54.359507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:54.373812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:54.375444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:54.484065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 640/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:30:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:30:54.490628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:54.598604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c685-dde9-477b-a4c6-e28ee422648a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:54.612583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:54.635105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622095f0-3b00-4373-8f13-07810697050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:54.635731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622095f0-3b00-4373-8f13-07810697050b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:54.636006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622095f0-3b00-4373-8f13-07810697050b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:54.636274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622095f0-3b00-4373-8f13-07810697050b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:54.648849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622095f0-3b00-4373-8f13-07810697050b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:54.650309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622095f0-3b00-4373-8f13-07810697050b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-622095f0-3b00-4373-8f13-07810697050b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:54.664950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:54.666129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:54.667246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:54.667246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:54.667246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:54.667246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 616/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:54.672429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:54.769239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:54.782940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87a1d-1033-473d-a6d3-d058bbe76a05 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:54.818035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:54.818686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:54.818935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:54.819206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:54.820357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:54.820671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:54.822035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:54.833502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 641/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:54.840545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622095f0-3b00-4373-8f13-07810697050b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:54.950175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622095f0-3b00-4373-8f13-07810697050b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:54.988118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:54.988950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:54.989252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:54.989507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:54.997046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:54.998526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:30:55.011433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:55.017840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:30:55.018803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 617/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:30:55.019339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:55.131191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee5fb1-c89c-40e9-a85f-9a1ee12a962b None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:55.167849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:55.168460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:55.168743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:55.169017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:55.169715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:55.170026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:55.171539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef30484-bf6b-4144-8375-12adc38764a0 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:55.186094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 642/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:30:55.192585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:55.224132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:55.304995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575915a-54bd-4692-bbf0-963a5451192e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:55.342754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:55.343513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:55.343893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:55.344122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:55.344995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=eef6061176284f0d912e496511a3c27a) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:55.345324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:55.346813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca47-0e05-4add-bb9b-cc948eeb3ecb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:55.353119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 618/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:30:55 2022] GET /identity/v3/role_assignments?user.id=eef6061176284f0d912e496511a3c27a&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:57.225229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110652e-396d-43db-8a21-fffe0a403274 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:57.225786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110652e-396d-43db-8a21-fffe0a403274 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:57.225786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110652e-396d-43db-8a21-fffe0a403274 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:57.227759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 643/1261] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:30:57.232759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:57.233170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:57.233326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:57.250958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:30:57.252119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:30:57.315148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 619/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:30:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:30:57.580916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110652e-396d-43db-8a21-fffe0a403274 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:57.592200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:57.678480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110652e-396d-43db-8a21-fffe0a403274 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:57.711442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:57.712055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:57.712331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:57.712601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:57.722239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:30:57.726955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:57.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:57.729048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 644/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:57.733934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:57.751752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:57.829128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68bd27-a820-4730-a087-24fc30b87d43 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:57.861484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:57.862076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:57.862351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:57.862618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:57.863297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:57.863591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:57.864941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:57.873036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 620/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:57.878742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:57.943628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:57.976888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c874b3-d886-40a9-ba88-7c170eb64eef None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.011097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.011719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.011996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.012262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.023254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.024707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:58.039365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:58.040502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:58.046261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:58.046261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:58.046261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:58.046261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 645/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:58.046544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.136427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:58.155567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcb1dc-8498-4f71-87cc-6a2dd4e30323 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.187594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.188184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.188458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.188726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.189347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:58.189641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.191020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.202856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 621/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:30:58.208531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.305821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea2ee9-25c1-4701-b243-edb656d4f414 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.325360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:58.339727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.340340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.340618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.340888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.348248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.349669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:30:58.360565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:58.361659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:58.362619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 646/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:58.367422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.466203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9ca09-1ed7-41dd-a402-d7632c189bcd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.499147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.499745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.500022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.500293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.501012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:58.501320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.502747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.511637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:58.519996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 622/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:58.526385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.626164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eef90-69a7-4a30-9ae5-b16b9698cd36 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.659197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.659813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.660093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.660368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.670580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.672045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:30:58.686299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:58.692270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:30:58.693437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:30:58.693437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:30:58.693437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:58.693437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 647/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:58.693724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.792530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49307ce-03ac-49ca-b046-bd0a70004df8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.826444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.827061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.827337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.827603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.828226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:58.828528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.829874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.840573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 623/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:58.846200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:58.877088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:58.943382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef297561-d13d-43c9-ba27-38cbdd35b009 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:58.976097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:58.977238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:58.977512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:58.977777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:58.984210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:58.985711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:30:58.997412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:30:58.998340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:30:58.999290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 648/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:30:59.004355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:59.065594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:59.106937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c756-9b51-4786-8e5b-dd2e038b2878 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:59.139278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:59.139877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:59.140159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:59.140434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:59.141121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:30:59.141427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:59.142887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08707aa-9dac-4b97-ac8b-8235e54cb82a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:59.154541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 624/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:30:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:30:59.160516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:30:59.250282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:30:59.258978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ef406-1f00-485e-a526-625efc490653 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:30:59.291116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:30:59.291751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:30:59.292030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:30:59.292302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] PATH_INFO: `/v3/projects/f3afacb02be44c7c892269f21581ad37/users/eef6061176284f0d912e496511a3c27a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:30:59.294170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f3afacb02be44c7c892269f21581ad37, user_id=eef6061176284f0d912e496511a3c27a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:30:59.295607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f180d1f-d4a5-407f-a585-80e0c9afc0bc None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:30:59.308145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 649/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 24 07:30:59 2022] PUT /identity/v3/projects/f3afacb02be44c7c892269f21581ad37/users/eef6061176284f0d912e496511a3c27a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:01.167503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ebc1-6f1f-4935-8637-aff7c74cba32 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:01.168060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ebc1-6f1f-4935-8637-aff7c74cba32 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:01.168060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ebc1-6f1f-4935-8637-aff7c74cba32 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:01.169415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 625/1274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:01.174492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:01.174896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:01.175080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:01.193248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:01.194428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:01.259473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 650/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:31:01.526060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153ebc1-6f1f-4935-8637-aff7c74cba32 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:01.629710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153ebc1-6f1f-4935-8637-aff7c74cba32 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:01.647483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:01.663563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:01.664200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:01.664478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:01.664747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:01.674481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:01.679178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:01.680758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:01.681782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 626/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:01.686774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:01.764575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:01.782577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3369cf-de2e-4102-b329-d61752a0c5f7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:01.815183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:01.815779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:01.816059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:01.816331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:01.816992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:01.817291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:01.818649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:01.827190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 651/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:01.832774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:01.929488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba773d-a21c-4806-94b3-40de1e2c9ab0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:01.962432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:01.963059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:01.963343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:01.963615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:01.973742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:01.975212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:31:01.988699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:01.989739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:31:01.990673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:31:01.990673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:01.990673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:01.990673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 627/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:01.995805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:02.100376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215783a2-06ea-4fed-8897-061856110c07 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:02.133920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:02.134520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:02.134826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:02.135099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:02.135739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:02.136040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:02.137397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:02.148671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 652/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:02.154414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:02.185285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:02.252074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1705126-0bc2-4b51-8ea6-2c252a145d66 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:02.283946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:02.284555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:02.284831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:02.285099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:02.292694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:02.294112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:31:02.305474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:02.306541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:02.307507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 628/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:02.312286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:02.316115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:02.408678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729858d2-97cb-4e9d-b9ac-eb1695024347 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:02.440762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:02.441353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:02.441628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:02.441898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:02.442614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:02.442942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:02.444351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:02.460740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 653/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 24 07:31:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:02.467241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:02.556488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:02.564634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dc2cf-e244-4346-8be4-d1eced5cd753 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:02.597892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:02.598496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:02.598798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:02.599066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:02.662953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:02.664856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:02.748363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:02.758713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 629/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:31:02.764541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:02.825221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:02.863216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84a30-db18-48c7-8e59-512372072727 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:02.895857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:02.896473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:02.896752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:02.897025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:02.906658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:02.908122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:31:02.922765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:02.923946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:31:02.925127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:31:02.925127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:02.925127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:31:02.925127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 654/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:02.929385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:03.026512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653612b-f483-4fe0-a235-95f91ebd55ca None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:03.059698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:03.060292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:03.060570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:03.060841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:03.061470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:03.061774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:03.063152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:03.074137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 630/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 24 07:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:03.079796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:03.177765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529417f-c2e4-4a77-b8fc-8dd77f59b3ea None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:03.190838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:03.210800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:03.211958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:03.212236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:03.212510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:03.218990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:03.220429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:31:03.232199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:03.233243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: ERROR keystone.server.flask.application Aug 24 07:31:03.234347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 655/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 24 07:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:03.238982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:03.287015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:03.335261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a0759-e1aa-40f8-b2de-1b1d204fe7ad None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:03.367813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:03.368397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:03.368674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:03.368946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:03.369620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:03.369919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:03.371351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f750c1d-b589-42ee-a3ed-4a5bcc8d93c4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:03.382671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 631/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 24 07:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:03.388587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:03.486518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027dbb0-e7ba-4bfd-92c0-53bee7f7ace9 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:03.518902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:03.519496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:03.519776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:03.520074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:03.520797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] RBAC: Adding query filter params (role.id=7b7098a7aed4412faff0e2ec081a365a, scope.project.id=f3afacb02be44c7c892269f21581ad37, user.id=eef6061176284f0d912e496511a3c27a) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:03.521104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:03.522524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0d021-6235-45a3-a961-347372f03f9e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:03.528721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 656/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:03 2022] GET /identity/v3/role_assignments?user.id=eef6061176284f0d912e496511a3c27a&scope.project.id=f3afacb02be44c7c892269f21581ad37&role.id=7b7098a7aed4412faff0e2ec081a365a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:05.449178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437ae7-2028-4e68-bda2-824696ee90d7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:05.449757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437ae7-2028-4e68-bda2-824696ee90d7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:05.449757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437ae7-2028-4e68-bda2-824696ee90d7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:05.451105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 632/1288] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:05.456347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:05.456780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:05.456885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:05.475213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:05.476405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:05.506256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:05.543676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 657/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:31:05.811471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86437ae7-2028-4e68-bda2-824696ee90d7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:05.914148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86437ae7-2028-4e68-bda2-824696ee90d7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:05.924441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:05.947746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:05.948368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:05.948649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:05.949175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:05.949748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:05.951148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:05.959667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:05.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:05.962819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 633/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:31:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:05.967257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:06.072138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a96ea2-7d97-45bf-83b6-74548a7b07fc None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:06.106661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:06.107304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:06.107586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:06.107860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:06.108491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:06.108797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:06.110162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baef907-c850-46c5-b53c-fe49c9bc757d None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:06.114960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 658/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 24 07:31:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:06.121281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:06.221764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a3cb-09bd-4447-b3ed-e213ca268651 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:06.256525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:06.257122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:06.257403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:06.257679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:06.258927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:06.259235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:06.260603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba13614-1a93-4192-8517-0d01ec8e672a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:06.264987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 634/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 24 07:31:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:08.112665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c63bbf-853d-4de3-817a-2761a76228d0 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:08.113248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c63bbf-853d-4de3-817a-2761a76228d0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:08.113248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c63bbf-853d-4de3-817a-2761a76228d0 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:08.114534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 659/1293] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:08.120436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eed66-c1dc-430a-bb28-657c736d7db3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:08.120854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eed66-c1dc-430a-bb28-657c736d7db3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:08.120973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eed66-c1dc-430a-bb28-657c736d7db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:08.138366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-752eed66-c1dc-430a-bb28-657c736d7db3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:08.139614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-752eed66-c1dc-430a-bb28-657c736d7db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:08.167946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:08.203896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 635/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:31:08.470835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c63bbf-853d-4de3-817a-2761a76228d0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:08.501568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:08.570465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c63bbf-853d-4de3-817a-2761a76228d0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:08.604478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:08.605073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:08.605348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:08.605616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:08.606247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:08.607640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c079c1d-55f2-47e0-9ed4-5a86dc3a6551 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:08.621704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 660/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 24 07:31:08 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:10.489929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8a5-d63f-4a04-b69c-ed60e6334b77 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:10.490485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8a5-d63f-4a04-b69c-ed60e6334b77 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:10.490485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8a5-d63f-4a04-b69c-ed60e6334b77 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:10.491830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 636/1296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:10.497505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b484e1-e576-454f-acae-14b824403c15 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:10.497883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b484e1-e576-454f-acae-14b824403c15 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:10.498047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b484e1-e576-454f-acae-14b824403c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:10.516222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-91b484e1-e576-454f-acae-14b824403c15 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:10.517396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-91b484e1-e576-454f-acae-14b824403c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:10.589523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 661/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:31:10.860705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ba8a5-d63f-4a04-b69c-ed60e6334b77 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:10.959614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ba8a5-d63f-4a04-b69c-ed60e6334b77 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:10.993598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:10.994332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:10.994569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:10.994801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:10.995366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:10.996721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:11.005924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:11.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:11.007842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 637/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:31:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:11.013514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b484e1-e576-454f-acae-14b824403c15 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:11.035072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:11.124774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b484e1-e576-454f-acae-14b824403c15 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:11.158266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:11.158900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:11.159183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:11.159458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:11.160086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:11.160389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:11.162072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:11.167895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 662/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 24 07:31:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:11.173929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:11.204764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:11.270138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29384f9-2e04-48bf-8819-72fe2f334481 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:11.301823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:11.302406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:11.302681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:11.302974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:11.303662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] RBAC: Adding query filter params (interface=public, service_id=bf748417813f4c8d8da6a1b207be5eef, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:11.303961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:11.305304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213777c6-6db4-4857-8245-e1e24c69d531 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:11.310657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 638/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 24 07:31:11 2022] GET /identity/v3/endpoints?service_id=bf748417813f4c8d8da6a1b207be5eef&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:11.316254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:11.394386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:11.412619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df81640-1f21-48fd-a25b-98226cc76099 None admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:11.445164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:11.445758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:11.446038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:11.446308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:11.446928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:11.448293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b523ef-92e4-49bf-957f-e4445da7868e None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:11.455111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 663/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 24 07:31:11 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:31:13.308090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca0c8-6fd0-4239-b7dc-220b5d3b2abc None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:13.308639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca0c8-6fd0-4239-b7dc-220b5d3b2abc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:13.308639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca0c8-6fd0-4239-b7dc-220b5d3b2abc None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:13.309957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 639/1302] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:13.315376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:13.315761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:13.315931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:13.334486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:13.335699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:13.400419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 664/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:31:13.671578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca0c8-6fd0-4239-b7dc-220b5d3b2abc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:13.763361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:13.770147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca0c8-6fd0-4239-b7dc-220b5d3b2abc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:13.802029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:13.802639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:13.802939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:13.803207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:13.803772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:13.805114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:13.813976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:13.815145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Aug 24 07:31:13.816180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 640/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 24 07:31:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:13.822652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:13.896942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:13.919086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad1091-6895-4b07-b1f8-66759de50272 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:13.952565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:13.953183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:13.953464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:13.953740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] PATH_INFO: `/v3/services` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:13.954392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:13.954706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:13.956087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61889bfd-4c82-445d-9fa6-d0da1ed63201 None admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:13.961047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 665/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 24 07:31:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:13.966824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:14.063721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8561e57-ca87-40c7-a7af-822e239a660b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:14.095518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:14.096111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:14.096387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:14.096660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:14.097319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:14.098685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622901d-9918-48d0-b9c8-ddef276011a2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:14.117256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 641/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:31:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:23.958118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=30 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:24.113051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:17.251251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032022-3e7d-4e97-ba65-a05328e2796f None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.252323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032022-3e7d-4e97-ba65-a05328e2796f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.252828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11032022-3e7d-4e97-ba65-a05328e2796f None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.259823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0c4d-4150-41ee-8221-6717731b5014 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.260681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0c4d-4150-41ee-8221-6717731b5014 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.261162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0c4d-4150-41ee-8221-6717731b5014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.265917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 666/1307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:17.371402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e6cc0c4d-4150-41ee-8221-6717731b5014 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:17.372671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e6cc0c4d-4150-41ee-8221-6717731b5014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:17.511423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 642/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:17.822343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.822688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.822934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.824300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 667/1309] 10.222.0.42 () {54 vars in 965 bytes} [Wed Aug 24 07:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:17.832510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8704b15-47d8-4889-8e2c-378fe3979d66 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.832878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8704b15-47d8-4889-8e2c-378fe3979d66 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.833162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8704b15-47d8-4889-8e2c-378fe3979d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.935234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a8704b15-47d8-4889-8e2c-378fe3979d66 None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:17.937100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a8704b15-47d8-4889-8e2c-378fe3979d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:17.961141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:18.026434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 643/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Aug 24 07:32:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:18.035667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:18.037619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:18.156380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddc0d6-95ef-4491-ab65-a99e4c8cf279 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:18.234834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:18.235442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.235833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.236111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.237725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:18.299022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:18.358595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:18.360073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d02f-fc0b-4887-9efc-61ef06b1659a service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:18.380787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 668/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:20.604691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.605228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.605371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.606852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 644/1312] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:20.613307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d93e-06c2-4f55-a8ff-c0212b6dbf89 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.613841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d93e-06c2-4f55-a8ff-c0212b6dbf89 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.613841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d93e-06c2-4f55-a8ff-c0212b6dbf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.684703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ce52d93e-06c2-4f55-a8ff-c0212b6dbf89 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:20.685894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ce52d93e-06c2-4f55-a8ff-c0212b6dbf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:20.787289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 669/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:21.063710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.065304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:21.167770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5bdb4-03a3-4565-8993-7ec26bd591f5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.238921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.241549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.242176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.242649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.243138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.244940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:21.353805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.355277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2143-9e47-4a1b-a94a-a033f2ee3473 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.373698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 645/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:23.588295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.588777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.588868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.590186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 670/1315] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:23.596111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f32c-65e1-485c-8cc8-6f13540bb47c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.596494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f32c-65e1-485c-8cc8-6f13540bb47c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.596658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f32c-65e1-485c-8cc8-6f13540bb47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.659855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d7e0f32c-65e1-485c-8cc8-6f13540bb47c None None] MFA Rules not processed for user `57a045168aa44e0386b65064bd0ea153`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:23.661061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d7e0f32c-65e1-485c-8cc8-6f13540bb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:23.711162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.756297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 646/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:24.038394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:24.039964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:24.050112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:24.142306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c11a60-3d74-4d7e-af90-0a865bb71fa1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:24.214969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:24.215570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.215846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.216113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.217733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:24.279175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:24.325712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:24.327638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091dc66-1c47-48a7-ba40-edce7de2f3ff service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:24.345509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 671/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:26.758062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.758658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.758658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.759995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 647/1318] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:26.765667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bda7b-2805-4d04-b62b-88f26472ffc7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.766067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bda7b-2805-4d04-b62b-88f26472ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.766211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bda7b-2805-4d04-b62b-88f26472ffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.831654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-373bda7b-2805-4d04-b62b-88f26472ffc7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:26.833273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-373bda7b-2805-4d04-b62b-88f26472ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:26.930943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 672/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:27.213037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.214576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:27.314560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b71a1-6330-4769-a31f-f2a51e19c400 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.385124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.385716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.385992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.386262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.387875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:27.395590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.494824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:27.496223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d7f2a-c58a-4863-905b-34bb413b0b1d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:27.514596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 648/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:30.082389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.082974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.082974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.084320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 673/1321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:30.089621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e03c-e695-4f23-8860-e32df1a253a0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.090073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e03c-e695-4f23-8860-e32df1a253a0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.090361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e03c-e695-4f23-8860-e32df1a253a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.155398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e2a0e03c-e695-4f23-8860-e32df1a253a0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:30.156645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e2a0e03c-e695-4f23-8860-e32df1a253a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:30.214400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.253599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 649/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:32:30.539976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.541516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:30.562710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.650034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52615de2-69b9-4971-822c-4cedfa42b15c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.724165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.724786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.725066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.725335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.727066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:30.796503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.834985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:30.836415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554b04-bed5-42c1-b6e0-a9b4ce433b91 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:30.854090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 674/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:32:32.802515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.803046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.803046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.804428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 650/1324] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:32.809740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c5319-6c6f-4401-a36a-9d3af86baed2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.810190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c5319-6c6f-4401-a36a-9d3af86baed2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.810274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c5319-6c6f-4401-a36a-9d3af86baed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.875340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-594c5319-6c6f-4401-a36a-9d3af86baed2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:32.876542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-594c5319-6c6f-4401-a36a-9d3af86baed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:32.973155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 675/1325] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:33.260117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:33.261659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:33.372837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdde0c7-5a7d-4968-aa48-299f2f845881 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:33.445843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:33.446446 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:33.446748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:33.447021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:33.448597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:33.465764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:33.557282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:33.558740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977e28f-4cfa-4eb3-9852-d0c6a00a5532 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:33.578471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 651/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:35.538855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:35.539322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:35.539439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:35.540742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 676/1327] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:35.546130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951226-34b6-4359-9864-5bba5603c008 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:35.546547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951226-34b6-4359-9864-5bba5603c008 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:35.546779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951226-34b6-4359-9864-5bba5603c008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:35.622525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d5951226-34b6-4359-9864-5bba5603c008 None None] MFA Rules not processed for user `57a045168aa44e0386b65064bd0ea153`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:35.623794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d5951226-34b6-4359-9864-5bba5603c008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:35.689348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:35.729876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 652/1328] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:36.028320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:36.029949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:36.064103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:36.142324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd2c9-eb88-4b9d-9253-4ca90742ff67 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:36.215115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:36.215726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:36.216003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:36.216271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:36.217868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:36.297398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:36.329914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:36.331404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2dc1d-04ef-4607-bbef-b6072e1517da service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:36.349623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 677/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:38.496171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.496730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.496730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.498135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 653/1330] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:38.503395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bbc9c-399b-4ba8-9e2b-9e23dbc2067e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.503819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bbc9c-399b-4ba8-9e2b-9e23dbc2067e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.503972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bbc9c-399b-4ba8-9e2b-9e23dbc2067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.568925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-547bbc9c-399b-4ba8-9e2b-9e23dbc2067e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:38.570152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-547bbc9c-399b-4ba8-9e2b-9e23dbc2067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:38.672855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 678/1331] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:38.955755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.957623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:39.062080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53db223-3788-4f03-905d-e66239ffa88b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:39.134454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:39.135011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:39.135284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:39.135551 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:39.137143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:39.163934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:39.244716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:39.246154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848c18d-45ce-4e8e-bc67-7d85d7b9a785 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:39.264872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 654/1332] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:41.218369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.218926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.218926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.220196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 679/1333] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:41.225336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426374-a1bd-4df5-993b-228c5a13562c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.225743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426374-a1bd-4df5-993b-228c5a13562c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.225884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426374-a1bd-4df5-993b-228c5a13562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.291360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f4426374-a1bd-4df5-993b-228c5a13562c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:41.292570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4426374-a1bd-4df5-993b-228c5a13562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:41.365956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.388688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 655/1334] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:41.673108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.674676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:41.714451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.776042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3d5a-479c-45bd-bd9a-bc0a481da597 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.846899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.847501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.847777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.848044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.849626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:41.930355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.956145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:41.957837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1704c-3895-42b8-b88d-61d9024f9316 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:41.975347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 680/1335] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:44.216380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d562-652d-43f4-b06a-3b0cc136399c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.216915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d562-652d-43f4-b06a-3b0cc136399c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.217023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d562-652d-43f4-b06a-3b0cc136399c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.218322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 656/1336] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:44.223565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de64c-2404-436f-b598-5c95a9674ea8 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.224017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de64c-2404-436f-b598-5c95a9674ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.224121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de64c-2404-436f-b598-5c95a9674ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.290087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5b6de64c-2404-436f-b598-5c95a9674ea8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:44.291301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5b6de64c-2404-436f-b598-5c95a9674ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:44.386547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 681/1337] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:44.678747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.679028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.679300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.680643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 657/1338] 10.222.0.42 () {54 vars in 965 bytes} [Wed Aug 24 07:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:44.686521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346456-7e5f-4807-9bb7-594de316c76b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.686838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346456-7e5f-4807-9bb7-594de316c76b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.687121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346456-7e5f-4807-9bb7-594de316c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.740874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.762508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-54346456-7e5f-4807-9bb7-594de316c76b None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:44.763750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-54346456-7e5f-4807-9bb7-594de316c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:44.838585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 682/1339] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Aug 24 07:32:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:44.847965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.849530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:44.950949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8202-2920-4dbd-b81c-362c46aa198f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.022776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.023379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.023660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.023931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.025522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:45.061215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.133207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.134642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfb704-8cde-4834-b525-8600a865aad8 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.154942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 658/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:49.533417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.533945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.533945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.535376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 683/1341] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:49.540764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a86a4d-b47b-4a93-abc6-b741d5e35144 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.541187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a86a4d-b47b-4a93-abc6-b741d5e35144 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.541283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a86a4d-b47b-4a93-abc6-b741d5e35144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.609280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-71a86a4d-b47b-4a93-abc6-b741d5e35144 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:49.611048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-71a86a4d-b47b-4a93-abc6-b741d5e35144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:49.695622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.707292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 659/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:49.991890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.993443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:50.100443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6626f-2cd4-4748-b525-e36b92dec36e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.138270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.174402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.175039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.175322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.175598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.177203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:50.288413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.289764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe10296-4d15-488a-8ea0-42a8819860dc service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.308575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 684/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:52.888169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.888620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.888710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.890111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 660/1344] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:52.895924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2f47-a2f5-431b-8d27-e292fef02bca None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.896353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2f47-a2f5-431b-8d27-e292fef02bca None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.896551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2f47-a2f5-431b-8d27-e292fef02bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.962384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a90a2f47-a2f5-431b-8d27-e292fef02bca None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:52.963626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a90a2f47-a2f5-431b-8d27-e292fef02bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:52.982249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.065372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 685/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:53.349812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.351378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:53.454550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228d44-3974-4ef9-9a47-db18bd52807d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.534968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.535579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.535861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.536134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.537771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:53.584106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.651061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.652512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9015c5-c4f9-421b-abe6-774fa1c6b9da service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.671926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 661/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:32:57.519443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.519443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.519994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.521029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 686/1347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:57.526761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f18f-f7de-4ca1-9bbc-844a5e2d789b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.527541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f18f-f7de-4ca1-9bbc-844a5e2d789b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.527818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f18f-f7de-4ca1-9bbc-844a5e2d789b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.601589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1135f18f-f7de-4ca1-9bbc-844a5e2d789b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:57.602892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1135f18f-f7de-4ca1-9bbc-844a5e2d789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:57.704260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.707127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 662/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:32:57.991679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.993318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:58.102304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c6b52-1784-4e2a-bc78-501c88c43245 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.150130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.177566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.178172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.178451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.178745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.180352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:58.286807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.288241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18f16d-af5c-4992-8448-8baba3a7d2d8 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.306240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 687/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:03.164945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f775-6707-4d04-ad92-c66f0471b035 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.165416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f775-6707-4d04-ad92-c66f0471b035 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.165497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f775-6707-4d04-ad92-c66f0471b035 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.169206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 663/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:33:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:03.175300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a7ba6-23f5-48c7-a1ca-b2d43876fe1e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.175742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a7ba6-23f5-48c7-a1ca-b2d43876fe1e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.176622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a7ba6-23f5-48c7-a1ca-b2d43876fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.264938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-648a7ba6-23f5-48c7-a1ca-b2d43876fe1e None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:03.266287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-648a7ba6-23f5-48c7-a1ca-b2d43876fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:03.295330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.426994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 688/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:33:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:07.627265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5cea-aa50-408f-a68e-87635296844d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.627883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5cea-aa50-408f-a68e-87635296844d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.627883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5cea-aa50-408f-a68e-87635296844d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.629150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 664/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:07.634207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4971d52-bdab-4361-9fd3-bca08f064917 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.634644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4971d52-bdab-4361-9fd3-bca08f064917 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.634967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4971d52-bdab-4361-9fd3-bca08f064917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.716904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e4971d52-bdab-4361-9fd3-bca08f064917 None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:07.718144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e4971d52-bdab-4361-9fd3-bca08f064917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:07.793558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.831993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 689/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:33:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:09.261184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511175-b6dc-4174-a44c-d8753ec22009 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.261703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511175-b6dc-4174-a44c-d8753ec22009 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.261703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511175-b6dc-4174-a44c-d8753ec22009 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.262982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 665/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:09.267845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681ff06-ac9b-4b35-a253-2eb1d2b61ae3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.268880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681ff06-ac9b-4b35-a253-2eb1d2b61ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.269061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681ff06-ac9b-4b35-a253-2eb1d2b61ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.345584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7681ff06-ac9b-4b35-a253-2eb1d2b61ae3 None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:09.346835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7681ff06-ac9b-4b35-a253-2eb1d2b61ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:09.452509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 690/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:33:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:12.730545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1243-b23b-448c-b626-4e8649f0711b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.731123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1243-b23b-448c-b626-4e8649f0711b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.731123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1243-b23b-448c-b626-4e8649f0711b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.732491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 666/1356] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:12.737791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15df4-fdc0-4d11-9b7c-f9ca0ba24052 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.738201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15df4-fdc0-4d11-9b7c-f9ca0ba24052 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.738348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15df4-fdc0-4d11-9b7c-f9ca0ba24052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.780684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:12.828870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a2f15df4-fdc0-4d11-9b7c-f9ca0ba24052 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:12.830073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a2f15df4-fdc0-4d11-9b7c-f9ca0ba24052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:12.964091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 691/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:13.265125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.265633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.266168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.268425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 667/1358] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 24 07:33:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:13.275951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71df99-5dd2-4d39-a2aa-8e356f1b3d09 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.276172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71df99-5dd2-4d39-a2aa-8e356f1b3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.276493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71df99-5dd2-4d39-a2aa-8e356f1b3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.363039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-df71df99-5dd2-4d39-a2aa-8e356f1b3d09 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:13.364435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-df71df99-5dd2-4d39-a2aa-8e356f1b3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:13.422905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.501657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 692/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 24 07:33:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:13.509908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.511493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:13.675408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8bb81-4de3-4864-b749-f3da9695add6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.753416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.800996 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.802111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.802605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.803218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.805276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:13.899443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.901058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f25ac-22a5-4d92-ae86-6f3b57ba4268 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.919943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 668/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:16.058017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.058572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.058572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.061294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 693/1361] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:16.066250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cac45-2679-4e0b-9ec1-531799c7ebd3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.066684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cac45-2679-4e0b-9ec1-531799c7ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.066842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cac45-2679-4e0b-9ec1-531799c7ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.136174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d18cac45-2679-4e0b-9ec1-531799c7ebd3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:16.137385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d18cac45-2679-4e0b-9ec1-531799c7ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:16.179494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.250143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 669/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:16.541479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.545258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:16.725868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498f2c7-c78e-4733-bdda-7db64f63d59d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.735854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.847184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.847803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.848163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.848335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.849963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:16.959201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.961072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf00b-ed64-4897-bbe7-0c1e0065eb76 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.979834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 694/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:19.226921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81518c-ff0d-4c76-b490-55d97fe5b88f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.227938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81518c-ff0d-4c76-b490-55d97fe5b88f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.228432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81518c-ff0d-4c76-b490-55d97fe5b88f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.232151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 670/1364] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:19.236631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec238-da45-44a2-ac0c-54f9840c6ddc None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.238048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec238-da45-44a2-ac0c-54f9840c6ddc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.238176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec238-da45-44a2-ac0c-54f9840c6ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.264802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.322232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c9fec238-da45-44a2-ac0c-54f9840c6ddc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:19.323486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c9fec238-da45-44a2-ac0c-54f9840c6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:19.426198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 695/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:19.719265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.719615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.719947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.721526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 671/1366] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 24 07:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:19.727635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8b55-dc1f-426c-96fc-99246db21cd9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.727923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8b55-dc1f-426c-96fc-99246db21cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.728196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8b55-dc1f-426c-96fc-99246db21cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.809063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5dff8b55-dc1f-426c-96fc-99246db21cd9 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:19.810359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5dff8b55-dc1f-426c-96fc-99246db21cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:19.846743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.934136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 696/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 24 07:33:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:19.941931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:19.943639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:20.098956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b56e2b-b240-48a8-8069-0c2f8826268e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.120276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:20.224966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.225729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.226103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.226474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.228239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:20.323710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.325526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2554fde-d3d0-4984-9301-e091627f9c83 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.344541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 672/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:20.984786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251140-5def-4f8c-82a0-57da70d22b49 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.985302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251140-5def-4f8c-82a0-57da70d22b49 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.985302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251140-5def-4f8c-82a0-57da70d22b49 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.987551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 697/1369] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:20.992327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70120ca2-b3c4-412f-b5b8-fa9a8091977b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.992715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70120ca2-b3c4-412f-b5b8-fa9a8091977b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.993185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70120ca2-b3c4-412f-b5b8-fa9a8091977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.029034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.067628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-70120ca2-b3c4-412f-b5b8-fa9a8091977b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:21.068845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-70120ca2-b3c4-412f-b5b8-fa9a8091977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:21.169897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 673/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:21.759074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.759074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.759074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.761278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 698/1371] 10.222.0.42 () {54 vars in 964 bytes} [Wed Aug 24 07:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:21.764888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acbf20-81c5-4d18-bfb0-b5a48c2d6105 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.765379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acbf20-81c5-4d18-bfb0-b5a48c2d6105 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.765561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acbf20-81c5-4d18-bfb0-b5a48c2d6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.851199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-47acbf20-81c5-4d18-bfb0-b5a48c2d6105 None None] MFA Rules not processed for user `923067fd0c864ca9bb202ee9585062f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:21.852483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-47acbf20-81c5-4d18-bfb0-b5a48c2d6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:21.900941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.931678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 674/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Aug 24 07:33:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:21.939741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:21.942074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.052623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6acc6-8c76-4e71-8252-3221dc77765d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.109801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.148277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.148904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.149187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.149901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.151413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.279432 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.280946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9849a-3809-4018-b757-cf897ec10737 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.307322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 699/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:22.338323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.338609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.338932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.340169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 675/1374] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:22.345748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c01dc-20b5-49d8-bdac-efbe3fa4e3c3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.346026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c01dc-20b5-49d8-bdac-efbe3fa4e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.347074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c01dc-20b5-49d8-bdac-efbe3fa4e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.437407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a28c01dc-20b5-49d8-bdac-efbe3fa4e3c3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:22.438926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a28c01dc-20b5-49d8-bdac-efbe3fa4e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.464088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.556923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 700/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:22.843140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.844761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.999154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f593a43-2f7c-479b-8aa4-1e98adc13268 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.052156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:23.118060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.118992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.119277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.119579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.121232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:23.210413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.212302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c9ec7-c97a-485d-9bdb-43e8e25d3f5a service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.229768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 676/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:25.158040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.158509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.158670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.160874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 701/1377] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:25.166736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef352b8-c63d-4a07-a16b-cc2142ae992f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.167154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef352b8-c63d-4a07-a16b-cc2142ae992f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.167299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef352b8-c63d-4a07-a16b-cc2142ae992f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.367141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2ef352b8-c63d-4a07-a16b-cc2142ae992f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:25.368503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2ef352b8-c63d-4a07-a16b-cc2142ae992f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:25.375962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.466422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 677/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:25.760099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.762083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:25.945087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24172-4644-4db3-8096-6aa7ea4b86a1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.949054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.066827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.067362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.067637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.067915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.069495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:26.162651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.164275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e297e95-3c85-46be-ab28-abc847a87f67 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.182525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 702/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:28.139515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.139963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.140223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.141564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 678/1380] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:28.146578 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623f179-7f89-4193-98b6-c31f69bbddfd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.147039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623f179-7f89-4193-98b6-c31f69bbddfd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.147170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623f179-7f89-4193-98b6-c31f69bbddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.159143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:28.217884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9623f179-7f89-4193-98b6-c31f69bbddfd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:28.219149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9623f179-7f89-4193-98b6-c31f69bbddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:28.326239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 703/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:28.613737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:28.615401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:28.767103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:28.779465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44349f16-23d4-4480-b2ab-3f94b4719f55 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:28.903139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:28.903701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.904050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.904393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.906562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:29.026076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.027735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b27758-9c29-4097-b46b-bcb86e74ea7f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.038619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:29.047602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 679/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:31.355539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.356087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.356196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.357556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 704/1383] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:31.363100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25570d7a-f203-4e0f-89e1-92e05cac54f3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.364283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25570d7a-f203-4e0f-89e1-92e05cac54f3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.364283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25570d7a-f203-4e0f-89e1-92e05cac54f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.433789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-25570d7a-f203-4e0f-89e1-92e05cac54f3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:31.435627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-25570d7a-f203-4e0f-89e1-92e05cac54f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:31.540695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 680/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:31.833400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.835077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:31.929558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.993149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d616-fb05-4f64-902c-8560f29e7482 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.113161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.113805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.114085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.114361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.116008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:32.172612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.221495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.223837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02304b9f-b27b-46aa-8672-a54fc1c2ac9f service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.243624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 705/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:34.287818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.288516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.288516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.289837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 681/1386] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:34.295540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa712-62c1-49d1-95be-e0cb772ceda6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.296012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa712-62c1-49d1-95be-e0cb772ceda6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.296184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa712-62c1-49d1-95be-e0cb772ceda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.366137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-18efa712-62c1-49d1-95be-e0cb772ceda6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:34.367887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-18efa712-62c1-49d1-95be-e0cb772ceda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:34.473516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 706/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:34.781222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:34.782804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:34.864569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:34.948597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925426-0cfe-4cdd-a278-395949feade4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.071913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.072527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.072812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.073087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.074747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:35.114979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.171277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.173424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28badbee-c1bd-4241-91ff-80ea6a6cdbbc service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.196470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 682/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:33:37.143910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.145780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.145780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.146150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 707/1389] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:37.151072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33173-534f-4f27-aa53-353c84c1a30d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.151500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33173-534f-4f27-aa53-353c84c1a30d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.151605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33173-534f-4f27-aa53-353c84c1a30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.219344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-75d33173-534f-4f27-aa53-353c84c1a30d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:37.220467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-75d33173-534f-4f27-aa53-353c84c1a30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.318419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 683/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:33:37.606962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.608505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.640903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.721143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571fe1-38d7-43c7-8e8c-3a0fac40a001 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.721420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571fe1-38d7-43c7-8e8c-3a0fac40a001 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.721697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571fe1-38d7-43c7-8e8c-3a0fac40a001 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.723083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 684/1391] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:37.728615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6388-4cd0-4b3d-8d5e-ca9a2b1e1605 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.728906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6388-4cd0-4b3d-8d5e-ca9a2b1e1605 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.729185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6388-4cd0-4b3d-8d5e-ca9a2b1e1605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.745125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.758579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480372fb-64f2-4505-8e5d-761fa924aaed None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.797659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1d8e6388-4cd0-4b3d-8d5e-ca9a2b1e1605 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:37.798979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1d8e6388-4cd0-4b3d-8d5e-ca9a2b1e1605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.862902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.876051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.876661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.876936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.877210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.878892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.898926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 685/1392] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:33:37.971922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.973561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd2d99-f2e4-4dec-91c4-fea2046ded7c service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.991708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 708/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:38.204429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.204897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.205016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.206463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 686/1394] 10.222.0.42 () {54 vars in 967 bytes} [Wed Aug 24 07:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:38.211687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d573fa-dd60-4565-beff-60c85348b170 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.212112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d573fa-dd60-4565-beff-60c85348b170 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.212235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d573fa-dd60-4565-beff-60c85348b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.289027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-03d573fa-dd60-4565-beff-60c85348b170 None None] MFA Rules not processed for user `ea288fe3842541999821bd863ca9d0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:38.290227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-03d573fa-dd60-4565-beff-60c85348b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:38.344060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.368806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 709/1395] 10.222.0.42 () {58 vars in 1064 bytes} [Wed Aug 24 07:33:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:38.376736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.378354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:38.476516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.484156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5511475-4c51-498b-ba19-b9cc5b7cc326 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.557767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.558373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.558653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.558984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.560593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:38.674000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.675472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62ce4-7311-40df-9cbc-dcacc727bab6 service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.693867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 687/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:39.432575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c1b-1880-4e74-bbca-c6728cb56a29 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.433061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c1b-1880-4e74-bbca-c6728cb56a29 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.433061 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c1b-1880-4e74-bbca-c6728cb56a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.501492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e0b51c1b-1880-4e74-bbca-c6728cb56a29 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:39.502724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e0b51c1b-1880-4e74-bbca-c6728cb56a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:39.589469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 710/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:33:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:39.631599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.632077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.632267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.633684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 688/1398] 10.222.0.42 () {54 vars in 943 bytes} [Wed Aug 24 07:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:39.640068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228961b3-de61-4de3-aa59-61554ec9d064 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.640355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228961b3-de61-4de3-aa59-61554ec9d064 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.640630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228961b3-de61-4de3-aa59-61554ec9d064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.715773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-228961b3-de61-4de3-aa59-61554ec9d064 None None] MFA Rules not processed for user `5c30642f91c54258b4ceec9005408be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:39.716886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-228961b3-de61-4de3-aa59-61554ec9d064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:39.729736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:39.826032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 711/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Aug 24 07:33:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:33:39.835176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:39.836682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:39.861318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:39.944837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935c2e0-5a68-47fd-8368-287fd0db6ad6 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.945119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935c2e0-5a68-47fd-8368-287fd0db6ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.945396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935c2e0-5a68-47fd-8368-287fd0db6ad6 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.946926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 712/1400] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:39.954410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.954707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.954991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.981461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc683e-5968-4738-8c8e-0f79a1605cd7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:40.024179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:40.025510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.086678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:40.087479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.088092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.088365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.090001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.097137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:40.100931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:40.124065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 713/1401] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:33:40.173576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:40.175216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bcf5f-f6c7-4f6b-99ab-54eaa344d69d service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:40.184470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 689/1402] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:33:40.415637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:40.416860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.543891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd19c-afc2-4493-9a0c-1879de07bb15 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:40.658422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:40.687236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:40.690207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.690491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.690787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.693239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.796683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:40.798391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca058c4a-7fde-44ad-87ab-2ce8f163c6bb service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:40.822883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 714/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:42.848079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.848683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.848683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.850172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 690/1404] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:42.855598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dae44-6384-40c8-b9ae-f441d58a74ea None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.856041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dae44-6384-40c8-b9ae-f441d58a74ea None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.856134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dae44-6384-40c8-b9ae-f441d58a74ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.926173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7c9dae44-6384-40c8-b9ae-f441d58a74ea None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:42.927465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7c9dae44-6384-40c8-b9ae-f441d58a74ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:42.984782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:43.040674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 715/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:33:43.328828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:43.330742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:43.491427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010dffd-b6e0-4dab-991f-b6120ca1ae81 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:43.494469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:43.621876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:43.622938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:43.623408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:43.623883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:43.625529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:43.721453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:43.724912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf1205-d1ec-4704-85d6-67f041289e63 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:43.749098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 691/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:45.744492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.745034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.745034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.746392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 716/1407] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:45.751708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6e070-a81d-4f94-8b7a-8c8a1fd9587f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.752130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6e070-a81d-4f94-8b7a-8c8a1fd9587f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.752389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6e070-a81d-4f94-8b7a-8c8a1fd9587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.763291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.831133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ecc6e070-a81d-4f94-8b7a-8c8a1fd9587f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:45.832443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ecc6e070-a81d-4f94-8b7a-8c8a1fd9587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:45.946821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 692/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:33:46.255573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.257132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:46.417615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8593a0-6174-44a8-9574-109ac04ac170 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.457373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.544839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.545359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.545634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.545897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.547537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:46.662007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.663651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eca555-24d5-4193-9fab-408f40b56af4 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.682246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 717/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:33:55.945043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=62 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:56.679774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=82 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:02.151833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62cd8-ee29-4253-849f-c625979d7470 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.152405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62cd8-ee29-4253-849f-c625979d7470 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.152405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62cd8-ee29-4253-849f-c625979d7470 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.153744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 693/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Aug 24 07:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:02.158956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14f28-8e18-44ed-8a8a-f0eeb9ac0bfe None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.159380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14f28-8e18-44ed-8a8a-f0eeb9ac0bfe None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.159482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14f28-8e18-44ed-8a8a-f0eeb9ac0bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.233542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-17d14f28-8e18-44ed-8a8a-f0eeb9ac0bfe None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:02.234750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-17d14f28-8e18-44ed-8a8a-f0eeb9ac0bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:02.347838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 718/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 24 07:34:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:02.362025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55586d-b0a8-428f-95ef-60212d34df3c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.362303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55586d-b0a8-428f-95ef-60212d34df3c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.362572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55586d-b0a8-428f-95ef-60212d34df3c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.363929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 694/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed Aug 24 07:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:02.368455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912018f-301f-465e-86a5-e08ccf91adde None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.368734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912018f-301f-465e-86a5-e08ccf91adde None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.369003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912018f-301f-465e-86a5-e08ccf91adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.440113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:02.444831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0912018f-301f-465e-86a5-e08ccf91adde None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:02.446575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0912018f-301f-465e-86a5-e08ccf91adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:02.560399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 719/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 24 07:34:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:04.932237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.932664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.932803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:04.934156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:04.939483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a866a39-f9fd-4d09-bc3f-6c87a75b0bd9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.939940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a866a39-f9fd-4d09-bc3f-6c87a75b0bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.940038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a866a39-f9fd-4d09-bc3f-6c87a75b0bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:05.004959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0a866a39-f9fd-4d09-bc3f-6c87a75b0bd9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:05.006145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0a866a39-f9fd-4d09-bc3f-6c87a75b0bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:05.059738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:05.101689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:05.385953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:05.387591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:05.532821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61855019-9a99-4d9a-b544-db73b3573286 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:05.607671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:05.647863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:05.648398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:05.648674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:05.648944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:05.650526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:05.738247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:05.739877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3208d-12f8-4ff0-8902-4a420d44dcb1 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:05.757480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 696/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:06.642898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:06.643447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:06.643447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:06.644801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:06.650359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e8-c5cb-4e97-99ed-12a7db692f79 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:06.650823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e8-c5cb-4e97-99ed-12a7db692f79 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:06.650960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e8-c5cb-4e97-99ed-12a7db692f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:06.725239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ec4837e8-c5cb-4e97-99ed-12a7db692f79 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:06.727031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ec4837e8-c5cb-4e97-99ed-12a7db692f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:06.753574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:06.821892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:06.969463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:06.971030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:07.114134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3377-54fe-4dc2-b551-135ea5ae9a31 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:07.153451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:07.228525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:07.229063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:07.229343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:07.229615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:07.231253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:07.321227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:07.322948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec157e1-3f05-4faf-88bc-3ad94ef8f63a service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:07.341025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 722/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:08.219065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:08.219622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:08.219622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:08.220956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:08.225816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9303-5adc-4cda-a146-ff002a30f37b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:08.226253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9303-5adc-4cda-a146-ff002a30f37b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:08.226351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9303-5adc-4cda-a146-ff002a30f37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:08.280581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:08.302039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5efe9303-5adc-4cda-a146-ff002a30f37b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:08.303273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5efe9303-5adc-4cda-a146-ff002a30f37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:08.399177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:08.548961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:08.550408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:08.700367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e0b7-cdc3-42de-82cc-e120e9764227 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:08.712748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:08.811882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:08.812538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:08.812807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:08.813068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:08.814642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:08.901450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:08.903119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc9824-a2ff-479b-a425-0cb5d6a4fdbe service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:08.920832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 699/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:09.787509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:09.788097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:09.788097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:09.789996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:09.795066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b904f7-8c3d-465e-84b4-697eafc7df89 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:09.795458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b904f7-8c3d-465e-84b4-697eafc7df89 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:09.795610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b904f7-8c3d-465e-84b4-697eafc7df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:09.814932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:09.870957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-31b904f7-8c3d-465e-84b4-697eafc7df89 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:09.872189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-31b904f7-8c3d-465e-84b4-697eafc7df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:09.967060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:10.114554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:10.116125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:10.224529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:10.261144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e9f92-f256-44d9-bcad-e1f3fb196490 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:10.373189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:10.373794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:10.374079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:10.374352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:10.376063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:10.452617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:10.468349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:10.469955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ca752-2f8e-4bdc-8c66-6aa774f821e4 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:10.487616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 725/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:11.345722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:11.346230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:11.346230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:11.347559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:11.352980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0697-9e6b-4d4f-8dc1-62aa334acf08 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:11.353386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0697-9e6b-4d4f-8dc1-62aa334acf08 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:11.353523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0697-9e6b-4d4f-8dc1-62aa334acf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:11.426777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-322f0697-9e6b-4d4f-8dc1-62aa334acf08 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:11.427969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-322f0697-9e6b-4d4f-8dc1-62aa334acf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:11.525488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:11.675500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:11.677034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:11.765697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:11.826432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f0ab5-d441-41ed-90f9-9106fbc03d6c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:11.936759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:11.937281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:11.937556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:11.937823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:11.939436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:11.985806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:12.027077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:12.028670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac72dba-5a12-46be-b4d6-16068982e33f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:12.046114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 702/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:12.847107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:12.847657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:12.847657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:12.849009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:12.854407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e31f-654a-4485-8536-c0bfd1aa37ef None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:12.854885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e31f-654a-4485-8536-c0bfd1aa37ef None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:12.854990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e31f-654a-4485-8536-c0bfd1aa37ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:12.929164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5571e31f-654a-4485-8536-c0bfd1aa37ef None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:12.930490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5571e31f-654a-4485-8536-c0bfd1aa37ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:13.026639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:13.176830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.178289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:13.228082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.325176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523034c-5992-4094-b0e9-79c5f42ea211 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.438756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.439292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.439572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.439844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.441462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:13.448961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.531623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:13.533247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8611861-9091-4307-a9fd-94fcb1904868 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:13.550934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 728/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:14.349930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:14.350465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:14.350465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:14.351884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:14.356849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698d6bf-9538-4d01-9335-6671bb55fe6c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:14.357251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698d6bf-9538-4d01-9335-6671bb55fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:14.357399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698d6bf-9538-4d01-9335-6671bb55fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:14.432017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6698d6bf-9538-4d01-9335-6671bb55fe6c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:14.433141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6698d6bf-9538-4d01-9335-6671bb55fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:14.499951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:14.529662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:14.679375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:14.680933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:14.699238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:14.831548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87a566-de0d-492b-9525-c28734d802a2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:14.918898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:14.946025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:14.946631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:14.946935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:14.947206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:14.948812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:15.036911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:15.038441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f8bb2-3e49-4149-aedd-c47062759198 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:15.056695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 705/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:15.869797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:15.870350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:15.870350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:15.871722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:15.876713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077688c-b76c-4381-b4e8-da21feab8f34 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:15.877134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077688c-b76c-4381-b4e8-da21feab8f34 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:15.877241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077688c-b76c-4381-b4e8-da21feab8f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:15.950911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6077688c-b76c-4381-b4e8-da21feab8f34 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:15.952099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6077688c-b76c-4381-b4e8-da21feab8f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:15.992974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.047350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:16.197085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.198533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:16.342664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237677a-151e-4df7-becd-dd1361e28e2c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:16.395430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.455532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:16.456072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:16.456352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:16.456623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:16.458250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:16.546796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:16.548431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a45d-087d-4dc2-97aa-8655f9610293 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:16.566328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 731/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:17.370326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:17.370916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:17.370916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:17.372257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:17.377257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337597-1d28-4d6c-b386-610bca660575 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:17.377718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337597-1d28-4d6c-b386-610bca660575 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:17.377823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337597-1d28-4d6c-b386-610bca660575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:17.451149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-97337597-1d28-4d6c-b386-610bca660575 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:17.452344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-97337597-1d28-4d6c-b386-610bca660575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:17.454352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:17.546840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:17.700810 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:17.702336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:17.848091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f768-0e8e-405b-b773-529b53d656d3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:17.875717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:17.965409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab0abb-b384-4280-958a-23477b73779a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:17.965933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0abb-b384-4280-958a-23477b73779a service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:17.966475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0abb-b384-4280-958a-23477b73779a service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:17.966779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0abb-b384-4280-958a-23477b73779a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:17.968372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-acab0abb-b384-4280-958a-23477b73779a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:18.055521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0abb-b384-4280-958a-23477b73779a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:18.057113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0abb-b384-4280-958a-23477b73779a service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:18.075124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 708/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:19.886553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.887129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.887129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.888460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:19.893900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd2557-8c23-4e5a-82ad-c3ec3baadbae None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.894346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd2557-8c23-4e5a-82ad-c3ec3baadbae None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.894539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd2557-8c23-4e5a-82ad-c3ec3baadbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.934459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:19.959852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-07bd2557-8c23-4e5a-82ad-c3ec3baadbae None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:19.961071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-07bd2557-8c23-4e5a-82ad-c3ec3baadbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:20.058032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:20.342858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:20.344451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:20.446932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e3ca0-0f7a-41d3-81e0-4d85dd5a6d24 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:20.479227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:20.521535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:20.522146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:20.522425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:20.522778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:20.525057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:20.634856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:20.636304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d30-5cc6-4950-8889-32ef048f12d7 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:20.654821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 734/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:22.712103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:22.712634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:22.712754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:22.714085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:22.719564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012519fb-4caf-4eac-8c50-a6181cbfe58c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:22.720005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012519fb-4caf-4eac-8c50-a6181cbfe58c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:22.720114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012519fb-4caf-4eac-8c50-a6181cbfe58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:22.785391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-012519fb-4caf-4eac-8c50-a6181cbfe58c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:22.786597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-012519fb-4caf-4eac-8c50-a6181cbfe58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:22.799744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:22.883023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:23.168817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:23.170367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:23.276960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57e23c-9a8c-43a5-86ef-c1f6a8d48f41 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:23.288984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:23.352781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:23.353388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:23.353667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:23.353943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:23.355588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:23.464874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:23.466307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200d959-f4a9-416f-9b32-aa38c9660ecb service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:23.484787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 711/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:25.759619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:25.760188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:25.760188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:25.761506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:25.766671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c7ca-8054-471d-bc4a-66c532342507 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:25.767153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c7ca-8054-471d-bc4a-66c532342507 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:25.767258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c7ca-8054-471d-bc4a-66c532342507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:25.814623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:25.831887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3809c7ca-8054-471d-bc4a-66c532342507 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:25.833077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3809c7ca-8054-471d-bc4a-66c532342507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:25.927025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:26.207787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:26.209332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:26.318769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f040-198c-4b1c-95fe-406728da0d20 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:26.359163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:26.392617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:26.393222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:26.393499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:26.393770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:26.395409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:26.502076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:26.503460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255ac81-aa5f-4e26-85bf-d06c0ec11f6b service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:26.521625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 737/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:28.516435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:28.516977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:28.517075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:28.518402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:28.523936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b48-eadd-41fd-819b-d2dba10700d2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:28.524371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b48-eadd-41fd-819b-d2dba10700d2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:28.524517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b48-eadd-41fd-819b-d2dba10700d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:28.591732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-82b22b48-eadd-41fd-819b-d2dba10700d2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:28.593231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-82b22b48-eadd-41fd-819b-d2dba10700d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:28.613820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:28.689244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:28.974577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:28.976181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:29.078666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dd4a2-0484-4c78-850f-efdcbd1e879b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:29.098748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:29.151506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:29.152100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:29.152374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:29.152643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:29.154221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:29.261096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:29.262494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a14ea-1b58-4c74-95c7-7818122cc3e1 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:29.282885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 714/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:31.247739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:31.248243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:31.248243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:31.249563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:31.254903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83ddf-df8a-40b9-ad80-e1dd32312439 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:31.255325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83ddf-df8a-40b9-ad80-e1dd32312439 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:31.255453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83ddf-df8a-40b9-ad80-e1dd32312439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:31.317994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:31.320946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8fe83ddf-df8a-40b9-ad80-e1dd32312439 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:31.323313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8fe83ddf-df8a-40b9-ad80-e1dd32312439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:31.419368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:31.707711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:31.709255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:31.812968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a1a0-0717-40bf-b710-9e19729fd560 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:31.861757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:31.886156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:31.886896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:31.887189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:31.887457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:31.889048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:31.995801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:31.997240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b01ed-48ef-4ccf-940d-087ca53ab8be service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:32.015468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 740/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:33.957991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.958562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.958562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:33.959984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:33.965742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a64f6-40b8-43bd-80eb-6988f7de9ed8 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.966148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a64f6-40b8-43bd-80eb-6988f7de9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.966279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a64f6-40b8-43bd-80eb-6988f7de9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:34.033209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-cd6a64f6-40b8-43bd-80eb-6988f7de9ed8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:34.034421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-cd6a64f6-40b8-43bd-80eb-6988f7de9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:34.063335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:34.129721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:34.420329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:34.421942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:34.524947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be09645-bf60-461c-9d14-7868a5d5c7bb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:34.552079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:34.597776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:34.598378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:34.598678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:34.599079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:34.600664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:34.708141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:34.709541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde1ee-14fc-45ba-9983-714daff95873 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:34.729899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 717/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:36.703816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.704373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.704373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.705783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:36.711210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97fbef-d29b-41f4-aae6-b889aad132dd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.711613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97fbef-d29b-41f4-aae6-b889aad132dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.711750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97fbef-d29b-41f4-aae6-b889aad132dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.776867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ed97fbef-d29b-41f4-aae6-b889aad132dd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:36.778059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ed97fbef-d29b-41f4-aae6-b889aad132dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:36.787926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:36.877617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:37.156606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:37.158248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:37.261814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977c67b-0e73-4cf7-9343-892293d40a84 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:37.315477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:37.333912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:37.334530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:37.334862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:37.335136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:37.336783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:37.451833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:37.453199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c27bfd-bb8b-4c6a-aa5a-a6349edefd70 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:37.471249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 743/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:39.427078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.427664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.427664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.428976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:39.434638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8547d-b5ea-463d-b9fd-a50fcd2d514c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.435065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8547d-b5ea-463d-b9fd-a50fcd2d514c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.435234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8547d-b5ea-463d-b9fd-a50fcd2d514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.502284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a5b8547d-b5ea-463d-b9fd-a50fcd2d514c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:39.503537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a5b8547d-b5ea-463d-b9fd-a50fcd2d514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:39.541732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:39.600994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:39.885819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:39.887417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:39.989103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef8f-d87f-4d6a-83aa-ecda9d133b42 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:40.026920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:40.062957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:40.063562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:40.063843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:40.064123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:40.065731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:40.172308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:40.173694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c74853-a709-4571-83f9-25d425f84630 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:40.192198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 720/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:42.134538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:42.135631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:42.135631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:42.136957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:42.142302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cedea-c180-4530-b7ef-e7629aee4ab7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:42.142746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cedea-c180-4530-b7ef-e7629aee4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:42.142894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cedea-c180-4530-b7ef-e7629aee4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:42.208161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d44cedea-c180-4530-b7ef-e7629aee4ab7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:42.209391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d44cedea-c180-4530-b7ef-e7629aee4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:42.227865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:42.304758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:42.587965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:42.589581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:42.692830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dfe85-8638-48c3-91b6-81b3b69cfb84 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:42.757781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:42.765885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:42.766483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:42.766787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:42.767055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:42.768745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:42.877039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:42.878472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d8487-e410-4493-ba66-7597b4257298 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:42.895917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 746/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:44.804211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:44.804788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:44.804788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:44.806134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:44.811512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cce27-646e-4469-be0d-d4f8b53bea8c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:44.811955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cce27-646e-4469-be0d-d4f8b53bea8c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:44.812081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cce27-646e-4469-be0d-d4f8b53bea8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:44.879236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-722cce27-646e-4469-be0d-d4f8b53bea8c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:44.880523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-722cce27-646e-4469-be0d-d4f8b53bea8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:44.929127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:44.977616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:34:45.256523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:45.258074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:45.359087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbfa99-d0b6-41d8-9437-2b2ee65b3137 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:45.405768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:45.430840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:45.431441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:45.431716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:45.431984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:45.433572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:45.542504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:45.543892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ba97c-02f8-4d8c-8a6c-38c696fad607 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:45.562428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 723/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:34:54.975839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=22 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:55.560177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=66 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:04.976217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4f2fc-f5f9-46ea-8335-9b4a2d451774 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.976700 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4f2fc-f5f9-46ea-8335-9b4a2d451774 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.976804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4f2fc-f5f9-46ea-8335-9b4a2d451774 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:04.978170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 748/1471] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:04.983475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d7b9-d406-4f6a-a1a4-049a7ab545c8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.983947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d7b9-d406-4f6a-a1a4-049a7ab545c8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.984046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d7b9-d406-4f6a-a1a4-049a7ab545c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:05.050502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-43e7d7b9-d406-4f6a-a1a4-049a7ab545c8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:05.051741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-43e7d7b9-d406-4f6a-a1a4-049a7ab545c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:05.146479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 724/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:05.443306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:05.443589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:05.443863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:05.445190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 749/1473] 10.222.0.42 () {54 vars in 967 bytes} [Wed Aug 24 07:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:05.449966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef664263-0ca6-4b71-a313-771c27eb3cf1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:05.450257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef664263-0ca6-4b71-a313-771c27eb3cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:05.450535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef664263-0ca6-4b71-a313-771c27eb3cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:05.529575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ef664263-0ca6-4b71-a313-771c27eb3cf1 None None] MFA Rules not processed for user `ea288fe3842541999821bd863ca9d0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:05.531043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ef664263-0ca6-4b71-a313-771c27eb3cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:05.556061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:05.613241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 725/1474] 10.222.0.42 () {58 vars in 1064 bytes} [Wed Aug 24 07:35:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:05.620717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:05.622280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:05.724960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc122d-23f7-4882-a695-5eaf07d78994 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:05.799265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:05.799872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:05.800149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:05.800419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:05.802038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:05.857794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:05.913279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:05.914757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01683e5-fa7a-4f8b-9e7b-b2610dfbad79 service glance] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:05.932956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 750/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:07.920559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.921119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.921119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.922477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 726/1476] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:07.927934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b31c-98ea-412e-aff1-bd5c585c3340 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.928342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b31c-98ea-412e-aff1-bd5c585c3340 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.928492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b31c-98ea-412e-aff1-bd5c585c3340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.994415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-06c3b31c-98ea-412e-aff1-bd5c585c3340 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:07.995646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-06c3b31c-98ea-412e-aff1-bd5c585c3340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:08.092315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 751/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:08.361705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:08.363285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:08.487677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824d8d-6ebd-4058-9172-4de2ea7c7ff7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:08.536192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:08.581377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:08.581984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:08.582265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:08.582538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:08.583209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:08.585174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:35:08.585174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:35:08.585174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:35:08.586328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ec14d-40d0-42b3-89ce-4e307a432ee8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:08.605994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 727/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:08 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:35:10.538178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.538889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.538889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.540123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 752/1479] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:10.545882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0f9d-d3f5-45cd-be29-68590d069c87 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.546303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0f9d-d3f5-45cd-be29-68590d069c87 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.546424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0f9d-d3f5-45cd-be29-68590d069c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.615179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fe3d0f9d-d3f5-45cd-be29-68590d069c87 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:10.616393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fe3d0f9d-d3f5-45cd-be29-68590d069c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:10.711849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 728/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:11.000754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:11.002445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:11.012300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.149344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0e6e-b674-4395-a508-a06647b78ff2 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:11.227964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.260229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:11.260838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.261116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.261385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.263018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:11.352143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:11.353768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262aed1-d278-47eb-9f03-d980ffaf515b service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:11.371548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 753/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:13.436182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:13.436863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:13.436863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:13.438096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 729/1482] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:13.443731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548699-082e-43db-9bed-c9e2c04a0d29 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:13.444189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548699-082e-43db-9bed-c9e2c04a0d29 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:13.444317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e548699-082e-43db-9bed-c9e2c04a0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:13.509472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8e548699-082e-43db-9bed-c9e2c04a0d29 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:13.510678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8e548699-082e-43db-9bed-c9e2c04a0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:13.557166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:13.609461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 754/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:13.894189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:13.895812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:13.919008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:13.998262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0374bee-bca1-40b6-b865-9051a6cbc191 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.071253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.071861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.072142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.072414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.074041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:14.146005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:14.183278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:14.184740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4bdb-5c12-4d18-874a-ab9fc7425f23 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:14.203570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 730/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:16.173991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:16.174552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:16.174552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:16.175910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 755/1485] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:16.181283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ebd6-cdbd-4bfa-909d-2a9a93c083ed None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:16.181707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ebd6-cdbd-4bfa-909d-2a9a93c083ed None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:16.181823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ebd6-cdbd-4bfa-909d-2a9a93c083ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:16.247569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ab70ebd6-cdbd-4bfa-909d-2a9a93c083ed None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:16.248777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ab70ebd6-cdbd-4bfa-909d-2a9a93c083ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:16.349962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 731/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:16.635156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:16.636712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:16.739580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdf198-2d6f-4cf0-8110-a6ec3cb35637 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:16.811798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:16.813858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:16.814945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:16.815426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:16.815792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:16.817793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:16.925867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:16.927336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e9d2a-fb94-4a25-b381-9dbcd05d2055 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:16.945522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 756/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:19.007499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:19.007992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:19.008097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:19.009406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 732/1488] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:19.014814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c5c6-3510-4d60-a1ca-620ce70e3745 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:19.015292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c5c6-3510-4d60-a1ca-620ce70e3745 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:19.015398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c5c6-3510-4d60-a1ca-620ce70e3745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:19.080516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4e78c5c6-3510-4d60-a1ca-620ce70e3745 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:19.081719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4e78c5c6-3510-4d60-a1ca-620ce70e3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:19.135576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:19.180533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 757/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:19.474642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:19.476335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:19.512313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:19.588000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd89ac-7c8c-4ab0-86b4-104761d8c7c0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:19.660324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:19.660922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:19.661200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:19.661472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:19.663098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:19.742472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:19.771246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:19.772683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c77e9-e40d-40a8-af07-20d1779f423c service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:19.791263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 733/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:21.871244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.871802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.871802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.873170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 758/1491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:21.878504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d222b-bfdb-4ffe-b9ed-f1cb8afb3d90 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.878957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d222b-bfdb-4ffe-b9ed-f1cb8afb3d90 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.879080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d222b-bfdb-4ffe-b9ed-f1cb8afb3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.945080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f61d222b-bfdb-4ffe-b9ed-f1cb8afb3d90 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.946287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f61d222b-bfdb-4ffe-b9ed-f1cb8afb3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:22.042803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 734/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:35:22.332051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.333597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:22.435743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00885e57-942c-4e60-a081-3285b01a2076 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.507946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.508551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.508829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.509107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.511386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:22.520666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.625887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.627614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bac4c-64c2-49fb-ad7c-6fef6dabaae7 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.645926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 759/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:35:32.040971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=82 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:32.643695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=50 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:39.593108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.593597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.593731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.670348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:39.671584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:39.768931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 735/1494] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:36:39.777112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.777623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.777727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.970144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:39.971359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.066156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 760/1495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:36:40.070917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.072135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.083487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5ffa-a04a-4145-b5db-7fde02ea7052 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.143609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.177438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.178075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.178361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.178634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.179313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:40.179617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.181253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:40.181253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:36:40.181253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:36:40.182170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.193427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 736/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:40.201785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.211000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3ff60-d50e-48b3-959f-6e875aed75f4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.272018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.305322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.305928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.306207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.306483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.307148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.308846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:40.308846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:36:40.308846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:36:40.310038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.365214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 761/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:40.369361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.370909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.403725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9947c0-b6df-42be-9df0-38b20f4de6a8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.497311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.497912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.498188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.498458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.499117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.500124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:40.500124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:36:40.500124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:36:40.501481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.544235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 737/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:36:40 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:40.548299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.549811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.612254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749883ba-3087-4b27-939c-efa527d90f24 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.694166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.706044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.706653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.706967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.707239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.707884 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.708895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:40.708895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:36:40.708895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:36:40.709777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.723831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 762/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:36:40.727937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.729452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.740690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed054b4-f787-4b33-be46-bddac522d224 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.766175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.836284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.836817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.837096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.837369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.838014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.839055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:40.839055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:36:40.839055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:36:40.840006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e911119-abe9-4b9a-87e3-87acd1c3266a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.852555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 738/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:36:40.856866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.858433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.891332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e17bd-66a2-404e-a7ce-eb4004ef40eb admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.983704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.984355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.984633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.984903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] PATH_INFO: `/v3/projects/e5d1d12dc0b64050864146ebc8aad142/users/d8c4448ceafb47088bd87d1ad2f6c268/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.006149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d1d12dc0b64050864146ebc8aad142, user_id=d8c4448ceafb47088bd87d1ad2f6c268, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.007198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:41.007198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:36:41.007198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:36:41.009200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bf63a-f5a7-4d08-90ad-1737a4117cf9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.033672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 763/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:36:40 2022] PUT /identity/v3/projects/e5d1d12dc0b64050864146ebc8aad142/users/d8c4448ceafb47088bd87d1ad2f6c268/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:41.046061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a87b-9548-47f8-b49c-75f292ac215f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.046333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a87b-9548-47f8-b49c-75f292ac215f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.046603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a87b-9548-47f8-b49c-75f292ac215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.102359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d912a87b-9548-47f8-b49c-75f292ac215f None None] MFA Rules not processed for user `d8c4448ceafb47088bd87d1ad2f6c268`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:41.103591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d912a87b-9548-47f8-b49c-75f292ac215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.178938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 739/1502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:36:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:36:41.192338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.192619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.192892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.194211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 764/1503] 10.222.0.42 () {54 vars in 964 bytes} [Wed Aug 24 07:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:41.199066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01aa75-1964-4328-8e55-6f411fb4f6b9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.199349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01aa75-1964-4328-8e55-6f411fb4f6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.199623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01aa75-1964-4328-8e55-6f411fb4f6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.231083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.274880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cb01aa75-1964-4328-8e55-6f411fb4f6b9 None None] MFA Rules not processed for user `923067fd0c864ca9bb202ee9585062f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:41.276082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cb01aa75-1964-4328-8e55-6f411fb4f6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.352510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 740/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Aug 24 07:36:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:36:41.360287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.361848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.411654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.464954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece2b37-0760-4835-a26d-f9f22165b4da None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.537272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.537878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.538158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.538428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.540114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.638828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.640218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.643749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.659496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 765/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:36:41.704091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b67b-e419-4ce8-94ff-ada043755b70 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.704381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b67b-e419-4ce8-94ff-ada043755b70 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.704660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b67b-e419-4ce8-94ff-ada043755b70 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.705934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 741/1506] 10.222.0.42 () {52 vars in 830 bytes} [Wed Aug 24 07:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:41.709493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.710704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.810422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9533c-ec16-4aff-8b1d-340d12306577 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.902874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.903543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.903692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.903966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] PATH_INFO: `/v3/users/d8c4448ceafb47088bd87d1ad2f6c268` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.916014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c4448ceafb47088bd87d1ad2f6c268)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.917064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:41.917064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:36:41.917064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:36:41.918277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.927854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.983955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c4448ceafb47088bd87d1ad2f6c268'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:41.984575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c4448ceafb47088bd87d1ad2f6c268'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:41.985097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c4448ceafb47088bd87d1ad2f6c268'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:42.008592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c4448ceafb47088bd87d1ad2f6c268'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:42.017942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c4448ceafb47088bd87d1ad2f6c268'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:42.036775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 766/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:36:41 2022] DELETE /identity/v3/users/d8c4448ceafb47088bd87d1ad2f6c268 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:42.041688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.042210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.042210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.100487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:42.102215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:42.176774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.190655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 742/1508] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:36:42.200311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.201531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:42.300409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e3e74-d061-4262-82b8-3b84f093dc8d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.357246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.378185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.378825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.379128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.379394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.381026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:42.486951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.488316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25000a21-d82a-488b-b6eb-1484b725ba2d service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.506113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 767/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:36:42.891371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.892632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:42.903822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68e68c-eba8-488d-b229-6e47cefaa792 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:43.001542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:43.002173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.002451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.002752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] PATH_INFO: `/v3/projects/e5d1d12dc0b64050864146ebc8aad142` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.012521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d1d12dc0b64050864146ebc8aad142)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:43.013537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:36:43.013537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:36:43.013537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:36:43.014770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:43.071253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d1d12dc0b64050864146ebc8aad142', 'request_id': 'req-63e5a5e5-e70f-4080-addf-00464043f248'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:43.080477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d1d12dc0b64050864146ebc8aad142', 'request_id': 'req-63e5a5e5-e70f-4080-addf-00464043f248'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:43.093600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d1d12dc0b64050864146ebc8aad142', 'request_id': 'req-63e5a5e5-e70f-4080-addf-00464043f248'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:43.115907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] The token cache is being invalidate because project e5d1d12dc0b64050864146ebc8aad142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:36:43.116195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:43.116624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-63e5a5e5-e70f-4080-addf-00464043f248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:36:43.118615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 743/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:36:42 2022] DELETE /identity/v3/projects/e5d1d12dc0b64050864146ebc8aad142 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:52.504014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=62 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.113943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:11.857811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:11.858393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:11.858393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:11.859775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 768/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:11.864490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be43c2-6b35-46b6-bed3-93bb2bbec802 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:11.864948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be43c2-6b35-46b6-bed3-93bb2bbec802 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:11.865055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be43c2-6b35-46b6-bed3-93bb2bbec802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:11.946329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-10be43c2-6b35-46b6-bed3-93bb2bbec802 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:11.947640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-10be43c2-6b35-46b6-bed3-93bb2bbec802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:12.051774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 744/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:12.203401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:12.205563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:12.324504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad71022-99fd-4d68-9ac3-fe328da4c684 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:12.330509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:12.331207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:12.331491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:12.331763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:12.339949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:12.341543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c6e13-1125-43fe-b05d-5ff4bcce0017 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:12.351613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 769/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:13.180910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a199bb-fdc1-478c-9905-9034905984c2 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.181439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a199bb-fdc1-478c-9905-9034905984c2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.181439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a199bb-fdc1-478c-9905-9034905984c2 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.182817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 745/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:13.187567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb72c-3bd4-42e6-b395-01191d747684 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.187987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb72c-3bd4-42e6-b395-01191d747684 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.188091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb72c-3bd4-42e6-b395-01191d747684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.207431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-51ebb72c-3bd4-42e6-b395-01191d747684 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:13.208627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-51ebb72c-3bd4-42e6-b395-01191d747684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:13.216278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 770/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:13.368941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a199bb-fdc1-478c-9905-9034905984c2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:13.383942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a199bb-fdc1-478c-9905-9034905984c2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:13.390786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:13.391030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.391220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.391502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.400852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:13.402496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50ee34-5343-4625-ba4e-2151295bab74 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:13.412009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 746/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:13.785094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66b6a-4fd6-4c7d-b8fb-0337b1298c47 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.785307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66b6a-4fd6-4c7d-b8fb-0337b1298c47 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.785584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66b6a-4fd6-4c7d-b8fb-0337b1298c47 None None] PATH_INFO: `/v3` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.786900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 771/1517] 10.222.0.16 () {54 vars in 925 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:13.792914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01453a-ff6c-4295-9435-2c38d8c38a3c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.793195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01453a-ff6c-4295-9435-2c38d8c38a3c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.793468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01453a-ff6c-4295-9435-2c38d8c38a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.852903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ae01453a-ff6c-4295-9435-2c38d8c38a3c None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:13.854095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ae01453a-ff6c-4295-9435-2c38d8c38a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:13.870228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:13.923516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 747/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 24 07:38:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:17.661681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672714-6c21-4e35-8cea-59ae158251c9 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:17.662171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672714-6c21-4e35-8cea-59ae158251c9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:17.662288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672714-6c21-4e35-8cea-59ae158251c9 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:17.663600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 772/1519] 10.222.0.42 () {54 vars in 961 bytes} [Wed Aug 24 07:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:17.668558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde85c56-da4f-41e1-a03b-a0ecd98d8f5f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:17.668993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde85c56-da4f-41e1-a03b-a0ecd98d8f5f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:17.669097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde85c56-da4f-41e1-a03b-a0ecd98d8f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:17.688707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cde85c56-da4f-41e1-a03b-a0ecd98d8f5f None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:17.690425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cde85c56-da4f-41e1-a03b-a0ecd98d8f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:17.698647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 748/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Aug 24 07:38:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:17.706463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0672714-6c21-4e35-8cea-59ae158251c9 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:17.719701 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0672714-6c21-4e35-8cea-59ae158251c9 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:17.725278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:17.725869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:17.726148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:17.726421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:17.734617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:17.736234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b5c2a-9014-4228-8098-4bd56d2a5806 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:17.745777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 773/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:24.643546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c7432-6a48-4a8d-9525-e599c951a017 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:24.644030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c7432-6a48-4a8d-9525-e599c951a017 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:24.644135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c7432-6a48-4a8d-9525-e599c951a017 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:24.645462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 749/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:24.650408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0e01-40a9-4664-9df4-c5f2471d87b7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:24.650921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0e01-40a9-4664-9df4-c5f2471d87b7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:24.651025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0e01-40a9-4664-9df4-c5f2471d87b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:24.672773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d97e0e01-40a9-4664-9df4-c5f2471d87b7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:24.673977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d97e0e01-40a9-4664-9df4-c5f2471d87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:24.683748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 774/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:24.837149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c7432-6a48-4a8d-9525-e599c951a017 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:24.855146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c7432-6a48-4a8d-9525-e599c951a017 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:24.861332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:24.862339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:24.862859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:24.863368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:24.875163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:24.877389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0190abf-44f4-4367-865f-0c346fe7db47 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:24.888580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 750/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:25.708298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12a942-a2d7-43e0-867e-4100e67586bf None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:25.708834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12a942-a2d7-43e0-867e-4100e67586bf None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:25.708834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12a942-a2d7-43e0-867e-4100e67586bf None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:25.710211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 775/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:38:25.714991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55956f6a-7c12-421c-87a0-7c3e30a7a750 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:25.715438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55956f6a-7c12-421c-87a0-7c3e30a7a750 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:25.715542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55956f6a-7c12-421c-87a0-7c3e30a7a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:25.735041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-55956f6a-7c12-421c-87a0-7c3e30a7a750 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:25.736808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-55956f6a-7c12-421c-87a0-7c3e30a7a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:25.744698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 751/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:25.894826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12a942-a2d7-43e0-867e-4100e67586bf None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:25.909503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12a942-a2d7-43e0-867e-4100e67586bf None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:25.914896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:25.915479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:25.915751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:25.916027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:25.923939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:25.925508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbbcb7-36aa-430b-ac06-a0ee45ef8fab service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:25.935146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 776/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:29.640481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:29.641020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:29.641020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:29.642934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 752/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:29.648326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9140e-c546-4e96-865d-e41c28286d25 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:29.648721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9140e-c546-4e96-865d-e41c28286d25 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:29.648876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9140e-c546-4e96-865d-e41c28286d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:29.668514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-91d9140e-c546-4e96-865d-e41c28286d25 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:29.669706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-91d9140e-c546-4e96-865d-e41c28286d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:29.677461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 777/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:29.963165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:29.964858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:30.043790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fe197-0d85-47c4-8c64-5cf5354a9323 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:30.047639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:30.048845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:30.049197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:30.049519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:30.058813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:30.060240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe7dfc-1f07-40bd-b26e-3f54b4139248 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:30.069473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 753/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:31.970026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6ab9-330c-48b5-aa52-437d42c5d099 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:31.970590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6ab9-330c-48b5-aa52-437d42c5d099 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:31.970590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6ab9-330c-48b5-aa52-437d42c5d099 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:31.971934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 778/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:31.977553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef0c19-12cb-4a79-95ef-43ec45a885ca None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:31.977932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef0c19-12cb-4a79-95ef-43ec45a885ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:31.978107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef0c19-12cb-4a79-95ef-43ec45a885ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:31.996994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7aef0c19-12cb-4a79-95ef-43ec45a885ca None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:31.998182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7aef0c19-12cb-4a79-95ef-43ec45a885ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:32.006315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 754/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:32.287041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc6ab9-330c-48b5-aa52-437d42c5d099 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:32.297926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc6ab9-330c-48b5-aa52-437d42c5d099 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:32.301511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:32.302109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:32.302387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:32.302660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:32.311847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:32.313775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667f700-f5f9-422a-bf70-51f9fd9df555 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:32.323398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 779/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:35.794211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:35.794794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:35.794794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:35.796182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 755/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:35.801549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd97d4-a639-4941-b028-bba962349c82 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:35.801954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd97d4-a639-4941-b028-bba962349c82 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:35.802184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd97d4-a639-4941-b028-bba962349c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:35.810002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=81 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:35.822663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-57fd97d4-a639-4941-b028-bba962349c82 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:35.823903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-57fd97d4-a639-4941-b028-bba962349c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:35.832187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 780/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:36.381612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:36.383315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:36.449715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:36.463267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f73d6-a66d-42eb-84b6-5bdb15ba0323 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:36.466815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:36.467417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:36.467690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:36.467960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:36.476750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:36.478173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fe640-5ea2-44c4-8839-db473ea36db8 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:36.487955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 756/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:38.421977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:38.422816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:38.422816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:38.423945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 781/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:38.430057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde64d2-23b9-41ff-bddb-d40b70d78b05 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:38.430510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde64d2-23b9-41ff-bddb-d40b70d78b05 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:38.430616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde64d2-23b9-41ff-bddb-d40b70d78b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:38.449654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8cde64d2-23b9-41ff-bddb-d40b70d78b05 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:38.450897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8cde64d2-23b9-41ff-bddb-d40b70d78b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:38.458674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 757/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:38:39.028454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:39.030167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:39.043717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12efb18-628f-4731-a69b-60ddb5488cba None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:39.047301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:39.047891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:39.048170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:39.048438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:39.057657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:39.059129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c40a5-8387-4e5a-928d-b3c8230f7fe8 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:39.069455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 782/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:38:48.444565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=11 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:49.064953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=8 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.467148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169eec-f6f1-4b66-ac5b-0c486ccc11e6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.467838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169eec-f6f1-4b66-ac5b-0c486ccc11e6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.467838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169eec-f6f1-4b66-ac5b-0c486ccc11e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.472211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92dd7-3f54-4130-bb7b-a1db040bbb98 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.475196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92dd7-3f54-4130-bb7b-a1db040bbb98 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.475196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92dd7-3f54-4130-bb7b-a1db040bbb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.501402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b3c92dd7-3f54-4130-bb7b-a1db040bbb98 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.502651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b3c92dd7-3f54-4130-bb7b-a1db040bbb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.503886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-63169eec-f6f1-4b66-ac5b-0c486ccc11e6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.505244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-63169eec-f6f1-4b66-ac5b-0c486ccc11e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.519030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 758/1540] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.524986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 783/1541] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.528225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.528511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.528789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.533749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.534272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.534560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.555951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.557151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.573363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 759/1542] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.574677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.576211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.578824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.597605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76c93-b397-4d0e-ae4b-48e4e710f40a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.604928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.605619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.605953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.606231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.606918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:00.607221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.607401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 784/1543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:00.608833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689d99-45c6-4739-bae5-9b907e96a614 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.612426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.630074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94db9-5207-4926-ab3d-57e81ca3332b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.633281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 760/1544] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:40:00.634796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.635051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.635274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.645485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.646110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.646388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.646987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.647649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:00.648065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.649726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:00.649726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:40:00.649726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:40:00.650000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac278dd-45f5-41d3-b649-c3c6d9ee44fe admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.667813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.669034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.675770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 785/1545] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:40:00.677024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.677317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.677849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.696540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 761/1546] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.697192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.712377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0b1db-7706-4725-8500-311a27797474 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.722495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.722936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.723071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.723331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.723598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.724227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.725243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:00.725243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:40:00.725243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:40:00.725852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.726435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44bcc2-bde4-4c1a-8570-a21357763ae4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.742757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 786/1547] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.743577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.754375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48c059-543d-4d1a-9c49-74df452a8942 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.759802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.760414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.760688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.760966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.761719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.763664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a47ee-43b3-4fad-8949-2a3f4acbcba0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.781660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 762/1548] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/projects => generated 355 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:00.782397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bd0f-699d-421e-a067-5dad50c2c189 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.782755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bd0f-699d-421e-a067-5dad50c2c189 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.784086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bd0f-699d-421e-a067-5dad50c2c189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.798374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 787/1549] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:00.806052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a742-bf2a-4cca-b4da-31b665482192 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.806052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a742-bf2a-4cca-b4da-31b665482192 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.806052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a742-bf2a-4cca-b4da-31b665482192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.807641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e291bd0f-699d-421e-a067-5dad50c2c189 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.808915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e291bd0f-699d-421e-a067-5dad50c2c189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.824426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-78a6a742-bf2a-4cca-b4da-31b665482192 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.825767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-78a6a742-bf2a-4cca-b4da-31b665482192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.870963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 763/1550] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.872018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b36a8-1edd-4237-a3eb-2f2b5c264c3b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.872306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b36a8-1edd-4237-a3eb-2f2b5c264c3b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.872588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b36a8-1edd-4237-a3eb-2f2b5c264c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.886182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 788/1551] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.887549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.887840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.888116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.892385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-497b36a8-1edd-4237-a3eb-2f2b5c264c3b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.893907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-497b36a8-1edd-4237-a3eb-2f2b5c264c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.908837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.910306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.910448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 764/1552] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.911603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.911890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.912169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.924133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 789/1553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.924133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.931634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8f57-3d50-4e9a-bf13-c1e363d7ff04 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.936285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.936880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.937161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.937284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.937434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.938084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.938417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.939212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:00.939212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:40:00.939212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:40:00.940416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017928eb-6856-4d39-a850-882c41c9eb27 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.946593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 765/1554] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:00.947423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.957909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cb0aa-8359-4de4-a6e4-2b5dd529c9a7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.963010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.963604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.963891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.964163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.964788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.965984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 790/1555] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/users => generated 278 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:00.966810 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aa76f-c1ac-474b-b8e5-72bb4013b2a3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.967248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667000-6e98-4c2b-b72e-54b6a721a85e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.967528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667000-6e98-4c2b-b72e-54b6a721a85e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.967799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667000-6e98-4c2b-b72e-54b6a721a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.988251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-32667000-6e98-4c2b-b72e-54b6a721a85e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:00.989371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-32667000-6e98-4c2b-b72e-54b6a721a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:00.992822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 766/1556] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:00.992822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.992822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.992822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.999900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 791/1557] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.001088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.001367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.001636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.015424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.017203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.023729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.024933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.029129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 767/1558] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.029129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.036281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dc8a-0369-4ba3-b9e3-2d54e7063356 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.036655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 792/1559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.037595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.043980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.044569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.044876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.045165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.045786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.046081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.047655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aecb30-f694-4caf-879c-46d9a1b47e78 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.047873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55725323-c027-459d-9c20-75f2071f85c7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.052891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.053530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.053805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.054081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.054755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.055047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.056632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.061062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 768/1560] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.062280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.062563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.062872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.074187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 793/1561] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.074978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.085016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98417-5d55-44ca-bd67-1ca16b09df09 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.089698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.090310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.090585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.090883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.091188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.091510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.092389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.093080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92441e9-3f40-4482-ab15-f2a2ce1f6894 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.101142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 769/1562] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.101805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.107886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 794/1563] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.108580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.108862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.109128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.111668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dd7bc-bd66-4a93-9a3b-da470ad0dec8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.116329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.116934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.117210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.117484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.118287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.119926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.129099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.129912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 770/1564] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.130500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.130710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.139076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 795/1565] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.139912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.140038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974c47d-7e17-42b3-821c-b5f0983eceae admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.144988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.145575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.145854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.146131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.146802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.147102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.148702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.150177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfa942-bdf7-482b-8646-4e1c2cfbe417 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.154870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e756403-6340-4424-927d-26406594166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.155452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e756403-6340-4424-927d-26406594166e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.155731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e756403-6340-4424-927d-26406594166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.156004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e756403-6340-4424-927d-26406594166e admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.156643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e756403-6340-4424-927d-26406594166e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.157513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e756403-6340-4424-927d-26406594166e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.159111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e756403-6340-4424-927d-26406594166e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.161406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 771/1566] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.162175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.171590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cd7eb-e2c2-462a-bce6-77ce2d54dc3a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.179038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 796/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.179038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e756403-6340-4424-927d-26406594166e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.179260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.179260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.179260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.179260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.179581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.180455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.184852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e756403-6340-4424-927d-26406594166e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.189502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.190099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.190375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.190759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.191411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.193263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.195009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.209992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 772/1568] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.210663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.225381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 797/1569] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.225381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.228014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.279026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d4946-0f67-4eb9-8b01-d1e036d47d67 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.284159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.284833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.285111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.285383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.286024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.286324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.287988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.288124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d2497-8cd3-4d1e-8260-2a722b83eb94 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.292736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.293330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.293610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.293875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.294518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.296129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.299561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 773/1570] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.300544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.306413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 798/1571] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.309821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.317360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b057c55-cd6b-4202-a526-552f81d0ca31 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.319688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f566-ff73-42d1-8984-6da74a249d50 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.321917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.322508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.322820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.323094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.323732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.324615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.325221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.325326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.325503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.325781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.326654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:01.326996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.329599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.335894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 774/1572] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.339346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.350335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 799/1573] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:01.350335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.350623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aedbdf-6b4d-40ea-bd9f-7375fbb68b97 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.354450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e56a3-50a2-46ec-a15b-b643c26397b5 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.354837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.355437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.355722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.355997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.356615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.358531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.359124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.359710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.360026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.360301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.360913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.363027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.386719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 775/1574] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.387531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.389624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 800/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.390507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.455075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b310-72e0-4441-ab20-1e524136f3b2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.456403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b751-dfb1-4d41-99d0-41df7a2e2dd2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.459825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.460432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.460713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.460986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.461109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.461592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.461700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.461873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.462147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.462854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.463569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.464821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.485370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 776/1576] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.486246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.490190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 801/1577] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.490190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.496561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816751-51bb-4a59-aa45-c17bf7cd5ea9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.499686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870a7bc-1c84-4331-9b36-d2962bf705fc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.501823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.502422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.502721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.503003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.503742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.504862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.505492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.505681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.505792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.506029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] PATH_INFO: `/v3/projects/6677843a7adf46c09f48866baa371dda/users/024eff730dbd4df985368f24aaa157d4/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.522150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6677843a7adf46c09f48866baa371dda, user_id=024eff730dbd4df985368f24aaa157d4, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.524843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.533915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 777/1578] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.534820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.541180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 802/1579] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:01 2022] PUT /identity/v3/projects/6677843a7adf46c09f48866baa371dda/users/024eff730dbd4df985368f24aaa157d4/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:01.541973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.563479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.573510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.602622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eed1db-d0ce-45a8-8c80-338cfd082dc3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.607607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.608266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.608544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.608816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] PATH_INFO: `/v3/projects/8d4da50a4bc4464d90be0ca6bb0ca8f5/users/d2172e5e2f7a45ed8e7dab49be5fe7ab/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.615135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08c62-507d-41ca-b417-34b323e63db2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.619954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.620621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.620902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.621177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.621830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.623817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.624283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4da50a4bc4464d90be0ca6bb0ca8f5, user_id=d2172e5e2f7a45ed8e7dab49be5fe7ab, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.625296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:01.625296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:40:01.625296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:40:01.627313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.640442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 778/1580] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:01 2022] PUT /identity/v3/projects/8d4da50a4bc4464d90be0ca6bb0ca8f5/users/d2172e5e2f7a45ed8e7dab49be5fe7ab/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:01.641322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.654637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 803/1581] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.654637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.708469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68cb3-5899-40ed-b8e1-ad55aa45e61a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.713277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.713892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17a8c4-1748-4dae-a268-89550423e056 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.714025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.714292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.714756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.715388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.717333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.718500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.719124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.719908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.720177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.720789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.722687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.739973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 779/1582] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.740795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.748155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 804/1583] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.749162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.753163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832eb98-4796-4481-94d8-44928a453cfa admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.757892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.758485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.758793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.759065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.759189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846672-e596-478b-b59f-28a53e85967a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.759704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.761296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.763861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.764483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.764766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.765042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.765678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.767816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec43b-3e2e-46fc-8f3e-808bf8425c82 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.771948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 780/1584] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.772837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.781370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 805/1585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.781370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.781370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.781370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.786363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208947-b88c-4b93-a15f-199816d1430f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.790933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.791525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.791798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.792066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.792677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.794772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968c859-b2fd-49aa-ab92-39774aa1d90e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.808554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] MFA Rules not processed for user `024eff730dbd4df985368f24aaa157d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.809759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.817444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 781/1586] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.818677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.819234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.819234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.847638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] MFA Rules not processed for user `d2172e5e2f7a45ed8e7dab49be5fe7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:01.848857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:01.878106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 806/1587] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.878106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.886486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0d54e-3ea0-4c51-a3a7-219e5fde2a7e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.891570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.892203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.892477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.892749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.893376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.895317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.911354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.920766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 807/1588] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:01.921482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 782/1589] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:01.921636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.922298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.931931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:01.936877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924bc13-0edc-4b69-9763-0d467667f677 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.938315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9436-0547-4997-a875-4ac53f6f9adb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.942772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.943028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.943383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.943628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.943739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.943904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.944017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.944173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.944679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.944794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.946286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.946410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.956926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 783/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.960822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 808/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.960973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.961091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.970113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579fb7d-b988-4b62-8080-c03d8963de32 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.970265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b0ec-8775-4d84-be1d-1430e854b388 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:01.974797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.975118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:01.975388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.975665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.975769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:01.975939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.976041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:01.976224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:01.976575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.976839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:01.978205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.978399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.988336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 784/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.991409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 809/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:01.991533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.991647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:01.999869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead4d2-4baf-43da-83c2-341f4641679c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.000507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389770-88c1-47e4-95fb-4842e70fa0b4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.004914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.005072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.005504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.005629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.005791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.005909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.006067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.006167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.006684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.006826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.008289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.008605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.019421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 785/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.020208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.030214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718a6a7-5dab-4fb7-812d-14074ab70dd6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.035980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.036635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.036861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.037136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.037770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.041829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 810/1595] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:01 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:02.041978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.042065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.050541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 786/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.050541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.061519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2f6e-ad01-4105-a8bb-db74b13a128e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.100942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c972b3-2db0-4b83-aaff-c460364873fc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.104767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.105378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.105660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.105954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.119219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.119823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.120104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.120380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.121018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.122645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.132365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 787/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.133159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.145918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213cbdc-6b05-439c-ba11-4abd57e270a7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.151209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.152058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.152058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.152320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.152951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.154542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.165926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.167302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 788/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.170687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.170826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.177254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707314-a17e-4fa0-9b45-d398a8d0eabf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.177687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 811/1599] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:02.178301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.182078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.182736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.183024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.183335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] PATH_INFO: `/v3/projects/3e9b29dbfd404c008fe871808d41e059/users/072548785a38488db94ee05dfbf8c8ec/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.189246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2712-7ccb-4c8f-9a0c-70d88ba7e0e2 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.193902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.194546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.194865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.195204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] PATH_INFO: `/v3/projects/7ca17bd7984241b1bbca7854ceb9440b/users/62897a3ee86d4dbbae38e801ad31ea07/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.199158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9b29dbfd404c008fe871808d41e059, user_id=072548785a38488db94ee05dfbf8c8ec, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.202253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.212210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca17bd7984241b1bbca7854ceb9440b, user_id=62897a3ee86d4dbbae38e801ad31ea07, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.215443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.217144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 789/1600] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/3e9b29dbfd404c008fe871808d41e059/users/072548785a38488db94ee05dfbf8c8ec/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.217959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.229946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 812/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/7ca17bd7984241b1bbca7854ceb9440b/users/62897a3ee86d4dbbae38e801ad31ea07/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.230757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.282263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:02.288993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb44319-e306-4f3b-9701-ab6ca26868c6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.293684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.294293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.294573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.294871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.295511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.296441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492c1fd-b0fc-4ea3-afbd-b9cdfaf43139 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.297160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.301381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.301999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.302285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.302856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.303549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.305605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.308582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 790/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.309915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.321403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0953e42-e142-44a1-9e1c-8ed2e4a97c6e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.326444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.327074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.327357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.327626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.328263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.330637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.330796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 813/1603] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:02.331371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.335780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:02.343119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c8f31-8856-445a-8468-5b27410052e7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.347182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 791/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.348235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.348399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.349175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.349455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.349731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] PATH_INFO: `/v3/projects/c128cf48470d4bcd91059b10fe41b68b/users/b30a8ed4973a4649baccb9963ceb84cb/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.358629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc326c8-7045-47f6-a43d-1cf9c295a44a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.363366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.364032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.364453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.364742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] PATH_INFO: `/v3/projects/0490376e47b647c8b8cad3ced5f4cfbb/users/59d4f237cc314a47bbae3db3429c961d/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.368135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c128cf48470d4bcd91059b10fe41b68b, user_id=b30a8ed4973a4649baccb9963ceb84cb, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.370949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181660c-4058-4f89-b0dc-123a9f19c997 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.384825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0490376e47b647c8b8cad3ced5f4cfbb, user_id=59d4f237cc314a47bbae3db3429c961d, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.387725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 814/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/c128cf48470d4bcd91059b10fe41b68b/users/b30a8ed4973a4649baccb9963ceb84cb/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.387869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824d441-da96-4c85-8554-09aecfbe59ee admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.394868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.395659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.396256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.404566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 792/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/0490376e47b647c8b8cad3ced5f4cfbb/users/59d4f237cc314a47bbae3db3429c961d/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.406218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.406507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.406817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.420195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:02.422565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:02.436057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:02.437475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:02.497143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 815/1607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:02.497984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.508237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 793/1608] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:02.509478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806f090-2b3f-428a-984d-5bccd1909b23 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.509868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.522585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981ad10-8020-48d0-95f5-3e4a69591eae None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.523297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.524039 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.524255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.524525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] PATH_INFO: `/v3/projects/7d34a03db7bc4ceca0ba091c71ed9719/users/a9ecbdaf05134e92890b15f551eca632/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.527296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.528051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.528341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.528610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.529256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.531117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.540341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d34a03db7bc4ceca0ba091c71ed9719, user_id=a9ecbdaf05134e92890b15f551eca632, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.545706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 794/1609] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:02.545841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d860a-656a-4e8a-b243-36c2d98ba3e8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.545957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.557520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7ba3b-ec44-4476-a9d9-f663574caa1e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.560260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 816/1610] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/7d34a03db7bc4ceca0ba091c71ed9719/users/a9ecbdaf05134e92890b15f551eca632/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.564541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.564825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.565098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.601323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] MFA Rules not processed for user `b30a8ed4973a4649baccb9963ceb84cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:02.603014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:02.634737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.635432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.635718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.635987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] PATH_INFO: `/v3/projects/d5672bdc355c4f63885138bd9c7f3497/users/ea609a2504e64f6f9380061080976a6e/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.655174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5672bdc355c4f63885138bd9c7f3497, user_id=ea609a2504e64f6f9380061080976a6e, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.658312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7be62-c230-4deb-8dc8-9b09e7c7da34 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.679037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 795/1611] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/d5672bdc355c4f63885138bd9c7f3497/users/ea609a2504e64f6f9380061080976a6e/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.679037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.679037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.679037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.681667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 817/1612] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:02.683921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.736584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] MFA Rules not processed for user `59d4f237cc314a47bbae3db3429c961d`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:02.739563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:02.755154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:02.781602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:02.814820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea7da-6989-4f75-964c-beb65f942bc8 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.825829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 796/1613] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:02.827490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.828945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:02.889448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.890192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.890646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.891021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.896588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a81e4-e93e-482f-8b70-d27cf9c7bdca None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.900543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.901463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.901741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.902013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.957814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.959581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.969160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 818/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:02.970527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:02.975265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:02.976734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ba161-f89a-4205-bbcf-45151393839f service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:02.979971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dddb-226a-42ac-bdef-44074c917f65 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:02.985867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:02.986586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.987226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.987226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] PATH_INFO: `/v3/projects/767a52e90c134821857fdc7a0d5560e6/users/be2bf90b865e4787854626088188c234/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.987827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 797/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:02.988976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a3a8a-40e3-4fcc-a86e-3473aefadca1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.989392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a3a8a-40e3-4fcc-a86e-3473aefadca1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.989607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a3a8a-40e3-4fcc-a86e-3473aefadca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.015792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767a52e90c134821857fdc7a0d5560e6, user_id=be2bf90b865e4787854626088188c234, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.018947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb55b6a-21b2-4e62-a307-0fb50b674758 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.023836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d49a3a8a-40e3-4fcc-a86e-3473aefadca1 None None] MFA Rules not processed for user `a9ecbdaf05134e92890b15f551eca632`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:03.025143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d49a3a8a-40e3-4fcc-a86e-3473aefadca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.042912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 819/1616] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:02 2022] PUT /identity/v3/projects/767a52e90c134821857fdc7a0d5560e6/users/be2bf90b865e4787854626088188c234/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:03.044225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.044451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.044672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.071594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:03.073411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.135013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 798/1617] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:03.137028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879c1c7-a3ed-45a8-96ff-ddbb4261fa45 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.137790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879c1c7-a3ed-45a8-96ff-ddbb4261fa45 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.137790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879c1c7-a3ed-45a8-96ff-ddbb4261fa45 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.139520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 799/1618] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 24 07:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:03.145313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879c1c7-a3ed-45a8-96ff-ddbb4261fa45 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.181554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.188463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 820/1619] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:03.188463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.192961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.198345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83189df-eb91-456c-9196-a8fb4bf8ba50 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.205795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.208966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.208966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.208966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.208966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.210882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae0667-a46f-44db-ab58-ca917568bddf admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.228934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 821/1620] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:03.232629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.232923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.233193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.280560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879c1c7-a3ed-45a8-96ff-ddbb4261fa45 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.288408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.289095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.289492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.289777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.290414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.292496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ba167-365c-454f-b8ec-5add498dce88 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.293186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:03.294368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.331586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 800/1621] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:03.333219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f1eb3-c35f-46cb-ab21-7d0bd46e9573 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.333699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f1eb3-c35f-46cb-ab21-7d0bd46e9573 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.334455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f1eb3-c35f-46cb-ab21-7d0bd46e9573 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.336840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 801/1622] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 24 07:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:03.337337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f1eb3-c35f-46cb-ab21-7d0bd46e9573 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.388797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 822/1623] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:03.389929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.441433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f1eb3-c35f-46cb-ab21-7d0bd46e9573 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.446511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.447246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.447528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.447808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] PATH_INFO: `/v3/projects/767a52e90c134821857fdc7a0d5560e6/users/be2bf90b865e4787854626088188c234/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.450072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767a52e90c134821857fdc7a0d5560e6, user_id=be2bf90b865e4787854626088188c234, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.452848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.475425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 802/1624] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:03 2022] PUT /identity/v3/projects/767a52e90c134821857fdc7a0d5560e6/users/be2bf90b865e4787854626088188c234/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:03.476715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.543870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d0887-0e2b-4b7f-bd8c-dd0f84941498 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.569506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc93c4-377d-4ee2-8aa3-551167566e17 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.581915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.582681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.583046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.583386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.584034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.586485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56c537-4efc-4c53-88fe-db519abc588c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.617709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.618973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.619746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.620106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.632710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.634215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.641595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 803/1625] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:03.641595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09bf62-f154-4ce6-86eb-afc80f4b7e6f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.641595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09bf62-f154-4ce6-86eb-afc80f4b7e6f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.641595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09bf62-f154-4ce6-86eb-afc80f4b7e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.648656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 823/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:03.649742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.664417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fd09bf62-f154-4ce6-86eb-afc80f4b7e6f None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:03.665949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fd09bf62-f154-4ce6-86eb-afc80f4b7e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.685665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.696958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.744712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73fac8-4207-4385-bc0e-0e30098c9fe3 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.751734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.752789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.753575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.753854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.778532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 804/1627] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:03.780234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.780753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.781216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.825850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] MFA Rules not processed for user `be2bf90b865e4787854626088188c234`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:03.829336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.833260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.835130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.862392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 824/1628] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:03.863422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.888586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63988c34-7932-4fb9-b261-7d46c855d433 service nova] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.901922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.902789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.904882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.905671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.908823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.912997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.940480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 825/1629] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:03.943973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.966569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0930cd-628d-4f3f-954a-e514c0b6d1c2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.990100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.993962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.995985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.996557 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.044860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 805/1630] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:04.046534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.087189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75447b-fa61-416c-9190-0337f9b3082e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.096852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.098994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.099821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.099821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] PATH_INFO: `/v3/projects/17c5254ee24e401a85d03f4415ce1e40/users/80344e8b91094ec0a6a73521f967551e/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.132923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c5254ee24e401a85d03f4415ce1e40, user_id=80344e8b91094ec0a6a73521f967551e, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.137610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.139743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.142036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.166277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 826/1631] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:04.166997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.181700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 806/1632] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:04 2022] PUT /identity/v3/projects/17c5254ee24e401a85d03f4415ce1e40/users/80344e8b91094ec0a6a73521f967551e/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:04.186574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.258273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9fa1e-6e71-413f-b40c-c0583a7e3997 service nova] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.274455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.325159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4280052-790b-4a10-9405-ed4a54e51bd1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.335414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.337240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.337526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.337801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.339834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.346807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e51c27-04aa-42fd-b4b3-330037490571 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.378360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 807/1633] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:04.380785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d37a7-0ba8-40f5-9160-6260c6959437 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.381156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d37a7-0ba8-40f5-9160-6260c6959437 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.381334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d37a7-0ba8-40f5-9160-6260c6959437 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.385316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.385937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.386214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.386484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.386765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 808/1634] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 24 07:40:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:04.387914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d37a7-0ba8-40f5-9160-6260c6959437 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.409560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d37a7-0ba8-40f5-9160-6260c6959437 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.417377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.418528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.418836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.419267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] PATH_INFO: `/v3/projects/17c5254ee24e401a85d03f4415ce1e40/users/80344e8b91094ec0a6a73521f967551e/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.421504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c5254ee24e401a85d03f4415ce1e40, user_id=80344e8b91094ec0a6a73521f967551e, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.425437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ae659-6b3a-4629-a75a-637479d7ac8f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.433184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.442171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 809/1635] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:04 2022] PUT /identity/v3/projects/17c5254ee24e401a85d03f4415ce1e40/users/80344e8b91094ec0a6a73521f967551e/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:04.443811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0911f6f-26ee-4222-9479-05022f94ace5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.444587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0911f6f-26ee-4222-9479-05022f94ace5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.444910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0911f6f-26ee-4222-9479-05022f94ace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.480942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a0911f6f-26ee-4222-9479-05022f94ace5 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:04.483355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a0911f6f-26ee-4222-9479-05022f94ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:04.545118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.547382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b38f3-408c-4cdf-b747-06c2691e2f19 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.565524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 810/1636] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:04.566806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 827/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:04.567154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.567443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.567718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.568838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899b148-49f3-4d03-904a-705647d7cf5e None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.569287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899b148-49f3-4d03-904a-705647d7cf5e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.570291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899b148-49f3-4d03-904a-705647d7cf5e None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.572767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 828/1638] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 24 07:40:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:04.581528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.582032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.582539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.619067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] MFA Rules not processed for user `80344e8b91094ec0a6a73521f967551e`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:04.620515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:04.628725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:04.631668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:04.658513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 829/1639] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:04.659417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.737374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 811/1640] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:04.741105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.787897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c3bd0-701a-4f28-863f-3896867f9e68 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.792832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.793429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.793714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.794196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.822057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.823674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.844013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 830/1641] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:04.881232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c16fbe-db90-443a-906f-8a78c137c8c0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.893548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.894178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.894460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.894763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.924770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.926856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.948993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 812/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:05.311814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:05.325461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7349-85a4-4657-a139-5ed6a4f32769 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:05.342536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:05.345073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:05.345553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:05.349163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:05.505962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:05.510892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:05.513698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:05.558972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 831/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:05.724562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:05.743495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02392b8-9322-44cd-8723-4d3928aeaa8e service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:05.759004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:05.759633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:05.759948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:05.760223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:05.789318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:05.791162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3c74f-9b47-46b4-839c-d4dae1d8c068 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:05.830651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 813/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:10.624668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:10.768651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d86be3-cda9-4ff6-8a34-9e1d39831021 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:10.787204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:10.787717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:10.788094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:10.788466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] PATH_INFO: `/v3/users/59d4f237cc314a47bbae3db3429c961d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:10.789766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d4f237cc314a47bbae3db3429c961d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:10.792137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:10.897905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d4f237cc314a47bbae3db3429c961d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:10.898895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d4f237cc314a47bbae3db3429c961d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:10.899769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d4f237cc314a47bbae3db3429c961d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:10.925253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d4f237cc314a47bbae3db3429c961d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:10.946396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d4f237cc314a47bbae3db3429c961d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:10.973263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 832/1645] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:10 2022] DELETE /identity/v3/users/59d4f237cc314a47bbae3db3429c961d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:10.977872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:10.978499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:10.982397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:11.020672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:11.033430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:11.037535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:11.191489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 814/1646] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:11.214953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:11.214953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:11.316906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:11.319523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:11.345025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea013e-86df-49d8-9eb9-b6b25240f8e0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:11.368617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:11.369222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:11.369509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:11.369792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] PATH_INFO: `/v3/users/a9ecbdaf05134e92890b15f551eca632` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:11.379900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ecbdaf05134e92890b15f551eca632)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:11.382910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:11.382910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Aug 24 07:40:11.382910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: Aug 24 07:40:11.384171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:11.388344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec64b3-f41b-4c84-8cdb-36cf5d00d9a2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:11.403688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:11.404336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:11.404626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:11.404899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:11.447408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:11.449740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbac1e-c112-404f-a99d-446d8b59b0f0 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:11.493040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 833/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:11.573594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ecbdaf05134e92890b15f551eca632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:11.574444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ecbdaf05134e92890b15f551eca632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:11.600396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ecbdaf05134e92890b15f551eca632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:11.643303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ecbdaf05134e92890b15f551eca632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:11.673017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7fe19a2c-0473-4dea-898c-fc8f502bf1a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ecbdaf05134e92890b15f551eca632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:11.705161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 815/1648] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:11 2022] DELETE /identity/v3/users/a9ecbdaf05134e92890b15f551eca632 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:11.706570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774278e9-3dbf-48a7-939e-40587797541a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:11.707790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774278e9-3dbf-48a7-939e-40587797541a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:11.708212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774278e9-3dbf-48a7-939e-40587797541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:11.783546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:11.783777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:11.784043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:11.789663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-774278e9-3dbf-48a7-939e-40587797541a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:11.794482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-774278e9-3dbf-48a7-939e-40587797541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:11.848666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:11.858503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] MFA Rules not processed for user `072548785a38488db94ee05dfbf8c8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:11.866631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:12.005681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 834/1649] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:12.023596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774278e9-3dbf-48a7-939e-40587797541a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:12.025361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-774278e9-3dbf-48a7-939e-40587797541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:12.126269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 816/1650] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:12.165392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:12.166866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:12.184402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:12.218932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774278e9-3dbf-48a7-939e-40587797541a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:12.229474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:12.233600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.233895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.237596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.268038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:12.278417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:12.315566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 835/1651] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:12.332592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:12.334883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:12.372068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543b868-2d2f-445e-8bf6-72b3a411c1b3 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:12.392414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:12.393366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.393654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.393935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] PATH_INFO: `/v3/projects/0490376e47b647c8b8cad3ced5f4cfbb` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.401127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0490376e47b647c8b8cad3ced5f4cfbb)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:12.403779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: WARNING py.warnings [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:40:12.403779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: warnings.warn(msg) Aug 24 07:40:12.403779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: Aug 24 07:40:12.405024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:12.413088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6ddba-ef9c-4ce7-8a81-686786793a11 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:12.429733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:12.431492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.431779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.432053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.455216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:12.458162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:12.511646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 817/1652] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:12.545373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0490376e47b647c8b8cad3ced5f4cfbb', 'request_id': 'req-d69f8625-6b88-49a1-91e5-e599e6ab3d66'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.573663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0490376e47b647c8b8cad3ced5f4cfbb', 'request_id': 'req-d69f8625-6b88-49a1-91e5-e599e6ab3d66'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.594414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0490376e47b647c8b8cad3ced5f4cfbb', 'request_id': 'req-d69f8625-6b88-49a1-91e5-e599e6ab3d66'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.633928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] The token cache is being invalidate because project 0490376e47b647c8b8cad3ced5f4cfbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:12.634236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.637184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.638791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 836/1653] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:12 2022] DELETE /identity/v3/projects/0490376e47b647c8b8cad3ced5f4cfbb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:12.641405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:12.663805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5bf48-b679-4987-be2d-3d74fc1b2d1c service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:12.833759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:12.834414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.834722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.835005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] PATH_INFO: `/v3/users/80344e8b91094ec0a6a73521f967551e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.836200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] RBAC: Authorizing `identity:delete_user(user_id=80344e8b91094ec0a6a73521f967551e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:12.839134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:12.965417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80344e8b91094ec0a6a73521f967551e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.966444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80344e8b91094ec0a6a73521f967551e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.977065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80344e8b91094ec0a6a73521f967551e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:12.978498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80344e8b91094ec0a6a73521f967551e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.002542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e908844b-325f-42f9-b6b0-98fba630dbee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80344e8b91094ec0a6a73521f967551e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.013844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.013844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:13.033552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f8625-6b88-49a1-91e5-e599e6ab3d66 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.047636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.048222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.048506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.048786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] PATH_INFO: `/v3/users/b30a8ed4973a4649baccb9963ceb84cb` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.049688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 818/1654] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:12 2022] DELETE /identity/v3/users/80344e8b91094ec0a6a73521f967551e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:13.050948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30a8ed4973a4649baccb9963ceb84cb)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.053405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.154643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:13.169937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30a8ed4973a4649baccb9963ceb84cb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.173539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30a8ed4973a4649baccb9963ceb84cb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.181921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30a8ed4973a4649baccb9963ceb84cb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.216527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30a8ed4973a4649baccb9963ceb84cb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.239688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30a8ed4973a4649baccb9963ceb84cb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.262068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 837/1655] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:13 2022] DELETE /identity/v3/users/b30a8ed4973a4649baccb9963ceb84cb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:13.267426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.269036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.270285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.326592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:13.330326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:13.382952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:13.408512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.411090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:13.584171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 819/1656] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:13.630087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacfdcc-acf9-4484-afa6-a934d6a0e635 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.636925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.641853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:13.662540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.664009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.664908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.667325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] PATH_INFO: `/v3/projects/7d34a03db7bc4ceca0ba091c71ed9719` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.683643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d34a03db7bc4ceca0ba091c71ed9719)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.685838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.824738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d34a03db7bc4ceca0ba091c71ed9719', 'request_id': 'req-fb916c32-d55f-44b5-b507-accec025e2a3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.844734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d34a03db7bc4ceca0ba091c71ed9719', 'request_id': 'req-fb916c32-d55f-44b5-b507-accec025e2a3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.867197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec9dfd-86d7-4e41-af2e-61432c6aa59d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.887734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d34a03db7bc4ceca0ba091c71ed9719', 'request_id': 'req-fb916c32-d55f-44b5-b507-accec025e2a3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.929512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] The token cache is being invalidate because project 7d34a03db7bc4ceca0ba091c71ed9719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:13.929702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.932176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-fb916c32-d55f-44b5-b507-accec025e2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:13.939100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 838/1657] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:13 2022] DELETE /identity/v3/projects/7d34a03db7bc4ceca0ba091c71ed9719 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:13.946621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.947692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.948153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.030946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:14.041058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:14.049075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.049822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.051338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.053869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.060632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:14.224615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:14.253126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.254821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef030d86-e7d1-43c0-a274-ac0935b6934d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.276923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 820/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:14.282958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 839/1659] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:14.288290 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.288869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.289222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.328912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:14.330243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:14.347801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 821/1660] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:14.351903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.367188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d3536-1cc8-4ece-8be1-a8a3e27e623c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.384363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.385458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.386462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.386959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.387840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:14.388333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.390667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.418925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 840/1661] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:14.422071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.440335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4ca89-f806-40b5-9388-8d01f500e052 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.453293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.454458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.454999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.455467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.456301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.460962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.488444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:14.528733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 822/1662] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:14 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:14.533597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.549386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402a01-e108-4956-8ff4-eea17d407a45 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.665311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.667316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:14.678766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.679521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.681097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.681483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.682714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.697334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.703011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd686c-90c6-4d24-a2e9-7dcd96c9504b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.723303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.724104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.724490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.727618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] PATH_INFO: `/v3/projects/17c5254ee24e401a85d03f4415ce1e40` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.737663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c5254ee24e401a85d03f4415ce1e40)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.742494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.787698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 841/1663] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:14 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:14.791671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.822230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337563-9361-4724-b6c9-22916ab56d1f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.830012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:14.830670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.830982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.831257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:14.831943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:14.833708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:14.858405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 842/1664] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:14.864148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:14.898596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c5254ee24e401a85d03f4415ce1e40', 'request_id': 'req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:14.912391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833317-fda0-4d63-8052-d03c43165fa0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:14.924300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c5254ee24e401a85d03f4415ce1e40', 'request_id': 'req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:14.933651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c5254ee24e401a85d03f4415ce1e40', 'request_id': 'req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:14.948698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] The token cache is being invalidate because project 17c5254ee24e401a85d03f4415ce1e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:14.950495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:14.950495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5e6b9d82-4223-4c2d-8988-0e5b6c5ce82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:14.952537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 823/1665] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:14 2022] DELETE /identity/v3/projects/17c5254ee24e401a85d03f4415ce1e40 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:14.960088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dee1a1-eda1-40ac-805d-a7065226c5a7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:14.960991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dee1a1-eda1-40ac-805d-a7065226c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:14.961503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dee1a1-eda1-40ac-805d-a7065226c5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.021859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a2dee1a1-eda1-40ac-805d-a7065226c5a7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:15.025683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a2dee1a1-eda1-40ac-805d-a7065226c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.104185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.106029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.106318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.107761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.108493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.112797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.152405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 843/1666] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:15.156718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.158899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.187533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209d10-f31b-46c5-a039-d49d165f4007 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.203375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.204628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.204914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.205255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] PATH_INFO: `/v3/projects/b2aeb9902d8541cb97f42ddf9dd1089e/users/24241700e02c4a68ad5a36287c1986c3/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.233161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2aeb9902d8541cb97f42ddf9dd1089e, user_id=24241700e02c4a68ad5a36287c1986c3, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.237519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b3914-6d7a-41e5-93cf-2bd5346a89ee admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.276176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 844/1667] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:15 2022] PUT /identity/v3/projects/b2aeb9902d8541cb97f42ddf9dd1089e/users/24241700e02c4a68ad5a36287c1986c3/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:15.277964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.278423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.278869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.299617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 824/1668] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:15.301339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.301647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.301953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.323261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] MFA Rules not processed for user `24241700e02c4a68ad5a36287c1986c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:15.325000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.340250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:15.361165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:15.363056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.482153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 845/1669] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:15.483148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.486848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 825/1670] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:15.489974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.490128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.515866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c469-4048-4e8f-bcd5-bd1e90f24010 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.520618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a20cf2-c5d3-44ca-81de-7eee4aae49a0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.531083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.531967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.532313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.532597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] PATH_INFO: `/v3/projects/c128cf48470d4bcd91059b10fe41b68b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.535180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c128cf48470d4bcd91059b10fe41b68b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.537277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.543205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.550444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.551121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.552401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.556466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.565286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.622677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:15.660331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c128cf48470d4bcd91059b10fe41b68b', 'request_id': 'req-a098ab1c-5be9-440d-b518-de361a9f1f77'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:15.666096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 826/1671] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:15.667287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.683469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c128cf48470d4bcd91059b10fe41b68b', 'request_id': 'req-a098ab1c-5be9-440d-b518-de361a9f1f77'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:15.707609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c128cf48470d4bcd91059b10fe41b68b', 'request_id': 'req-a098ab1c-5be9-440d-b518-de361a9f1f77'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:15.729773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] The token cache is being invalidate because project c128cf48470d4bcd91059b10fe41b68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:15.729998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:15.734519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:15.736982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 846/1672] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:15 2022] DELETE /identity/v3/projects/c128cf48470d4bcd91059b10fe41b68b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:15.737786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.746487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.810835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e401c6-339a-4fa5-86af-17a9460e8d62 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.821430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.822064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.822406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.822716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.823650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:15.823963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.827818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3303614-8dfc-4272-8a2e-c72435203f41 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.866824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 827/1673] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:15.870769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.871546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.871546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.926665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:15.932184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.942224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098ab1c-5be9-440d-b518-de361a9f1f77 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.960082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 828/1674] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:15.967509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.973462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.974284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.978966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.979378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.979759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.981195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.988320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee967c27-a0d9-4dd9-8566-e92e943e7fa6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.004242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27edba6-9537-42da-b02a-17efe788c569 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.011169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.011823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.012107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.012386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.013028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.021486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.063885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 847/1675] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/users => generated 296 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:16.066904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.067454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.067971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.104200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 829/1676] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:16.105388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.163870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:16.163870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:16.273897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:16.333187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a8feb-aa58-45c4-8a09-dc03d0d4d96a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.350730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.353122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.353408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.353682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.358193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.361287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.370424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 848/1677] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:16.372260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.387488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 830/1678] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:16.392500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.404684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dcd57-0187-4d3f-8454-2dfc965a2d03 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.422102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.423238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.423716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.424258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.425250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.430490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.434836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44dcde-0940-4262-9ad7-f132f01c4469 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.450680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.451974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.452423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.452853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.454467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:16.455142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.457289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f5d6-901a-43cd-a0c4-0f98ed37eb72 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.477588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 831/1679] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:16.488334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.488994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.489353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.499809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 849/1680] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:16.502518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.531309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6b427-b68e-4dcf-9c4f-b12348d8b8b4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.532596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] MFA Rules not processed for user `62897a3ee86d4dbbae38e801ad31ea07`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:16.533908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:16.541313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.542769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.543155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.544314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] PATH_INFO: `/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246/users/76672e39ff4e4842b392af17746a18dd/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.582921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:16.585827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a365ee7bbed4acd9a72c6eb2e49e246, user_id=76672e39ff4e4842b392af17746a18dd, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.609751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.645389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 850/1681] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:16 2022] PUT /identity/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246/users/76672e39ff4e4842b392af17746a18dd/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:16.646419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.682719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 832/1682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:16.685068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.710375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081bc9d-f0e7-4c35-8bb7-ed91591e1eed None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.789806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15098f2-6399-42f5-90f2-d25ec2dc2c58 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.809405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.810465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.810465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.810679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.814363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.818342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489d364-20f6-453e-acc4-60cd3fdd4f24 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.830030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.830572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.830952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.831291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.832075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.835379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.854857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 833/1683] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:16.854857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.883612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539288b-db83-4b43-9f86-e28b8ef8eb9b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.895263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 851/1684] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/projects => generated 367 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:16.896745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.896983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.897262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.899337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.900001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.900434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.900783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.901382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.903093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.933195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 834/1685] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:16.941060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.941389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] MFA Rules not processed for user `62897a3ee86d4dbbae38e801ad31ea07`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:16.942652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:17.007755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55c32e-99a1-4708-a4e2-d60e0bc9fbe2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.144951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 852/1686] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:17.148313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:17.203943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:17.204725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.205107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.205489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.206269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:17.208099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:17.213887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:17.232533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 835/1687] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:17.232533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:17.259756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6759-7afa-499d-83d0-1483f1f45413 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.271887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:17.272677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.272856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.273134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] PATH_INFO: `/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246/users/76672e39ff4e4842b392af17746a18dd/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.280177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a365ee7bbed4acd9a72c6eb2e49e246, user_id=76672e39ff4e4842b392af17746a18dd, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:17.288295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:17.326432 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92310d4-b9c9-42ce-b0b5-2a288708fa1b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.334072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 836/1688] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:17 2022] PUT /identity/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246/users/76672e39ff4e4842b392af17746a18dd/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:17.353241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:17.357239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:17.460495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:17.461132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.461411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.461684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.462530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:17.471750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:17.538237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 853/1689] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:17 2022] POST /identity/v3/users => generated 284 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:17.542653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:17.572428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:17.573360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26355953-dd19-41f4-87e5-9ec14a643f7a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.585905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:17.587007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.587291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.587567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] PATH_INFO: `/v3/projects/3744db727b864f949971cd8f70cb372a/users/31f7af920dda4877a5fd64699ec37ee7/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.636776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3744db727b864f949971cd8f70cb372a, user_id=31f7af920dda4877a5fd64699ec37ee7, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:17.642288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6b884-a71f-4ad7-a465-63b3c77a18f9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:17.661455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962ccda-576d-49b3-8714-6c5ed2c8a6a5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.672252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 854/1690] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:17 2022] PUT /identity/v3/projects/3744db727b864f949971cd8f70cb372a/users/31f7af920dda4877a5fd64699ec37ee7/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:17.673605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fff40-1daf-4435-b4fd-a899b8a63b4b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.673900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fff40-1daf-4435-b4fd-a899b8a63b4b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.676379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fff40-1daf-4435-b4fd-a899b8a63b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.679154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:17.679873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.680227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.680625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.733702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8c0fff40-1daf-4435-b4fd-a899b8a63b4b None None] MFA Rules not processed for user `76672e39ff4e4842b392af17746a18dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:17.735325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8c0fff40-1daf-4435-b4fd-a899b8a63b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:17.847343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:17.849097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:17.873949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 837/1691] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:17.875767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:17.884265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:17.909010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 855/1692] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:17.910903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:17.911203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:17.911535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:17.921294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d888b0-94e9-4d97-b9b7-efd284aac3cf service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:17.941371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] MFA Rules not processed for user `072548785a38488db94ee05dfbf8c8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:17.943357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:17.959487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:18.064483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 856/1693] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:18.066786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:18.192374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:18.194097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.195584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.196580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] PATH_INFO: `/v3/users/d2172e5e2f7a45ed8e7dab49be5fe7ab` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.199120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2172e5e2f7a45ed8e7dab49be5fe7ab)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:18.202751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:18.277923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118f0a0-6b54-4540-a7df-305e650eefb4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:18.291260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:18.292056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.292440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.292816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.293590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:18.295681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643c36-dd96-4310-8ad3-218141cb2c5b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:18.319006 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 857/1694] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:18.320617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.321295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.321662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.370252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2172e5e2f7a45ed8e7dab49be5fe7ab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:18.374468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2172e5e2f7a45ed8e7dab49be5fe7ab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:18.389112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] MFA Rules not processed for user `31f7af920dda4877a5fd64699ec37ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:18.390669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.395367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2172e5e2f7a45ed8e7dab49be5fe7ab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:18.399961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2172e5e2f7a45ed8e7dab49be5fe7ab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:18.429368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2172e5e2f7a45ed8e7dab49be5fe7ab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:18.433671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:18.454266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 838/1695] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:17 2022] DELETE /identity/v3/users/d2172e5e2f7a45ed8e7dab49be5fe7ab => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:18.455380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:18.461036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.573639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1d126-0588-4709-aab8-0d1277c62453 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:18.583363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:18.584090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.584445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.584788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.588182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.663744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 858/1696] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:18.664850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:18.669140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.715092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dd68f-41a1-45e1-bc5e-47f703404625 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:18.775855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:18.777326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696ba1-8eb2-4da2-82e3-6bc16d5ccb68 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:18.803152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 839/1697] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:18.804668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.804957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.805236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.820572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:18.849947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:18.851159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.880806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:18.881551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:18.883973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:18.884836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:18.885638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:18.889864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:18.912534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 859/1698] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:18.915479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:18.918949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:18.977836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 840/1699] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:18.979019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:19.013207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7c41-5270-41ff-8788-dc1f2dfd21b8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:19.042206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:19.042941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.043218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.043658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] PATH_INFO: `/v3/projects/f4d335b660ed43b099882d21b3c713c2/users/88224875ce204bdd97ce6a20f0df8fa3/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.089118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d335b660ed43b099882d21b3c713c2, user_id=88224875ce204bdd97ce6a20f0df8fa3, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:19.094113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:19.126321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf48c0-fbfc-4096-a13b-47ab4ec09d36 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:19.126641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 841/1700] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:18 2022] PUT /identity/v3/projects/f4d335b660ed43b099882d21b3c713c2/users/88224875ce204bdd97ce6a20f0df8fa3/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:19.126641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:19.135004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:19.221963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:19.291820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:19.293183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.294411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.294900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.332170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12a09-e147-4d6f-8bac-fca21ceb77c4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:19.345929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:19.346581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.349360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.357055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.410490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:19.412705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32a06-0d1c-4722-a3cb-0b045b532402 service glance] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:19.448014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 860/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:19.460956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916815b4-80f2-4f27-9ae7-80291fa4a61d None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.461274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916815b4-80f2-4f27-9ae7-80291fa4a61d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.461570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916815b4-80f2-4f27-9ae7-80291fa4a61d None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.467886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 861/1702] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 24 07:40:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:19.474050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d62-7b84-42f1-b828-76df9d33133d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.476463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d62-7b84-42f1-b828-76df9d33133d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.476860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d62-7b84-42f1-b828-76df9d33133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.550122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-54771d62-7b84-42f1-b828-76df9d33133d None None] MFA Rules not processed for user `88224875ce204bdd97ce6a20f0df8fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:19.551583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-54771d62-7b84-42f1-b828-76df9d33133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:19.576202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:19.579719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ba15-10a7-42c4-82f0-85e7a81a742a service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:19.595672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:19.616836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 842/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:19.618422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.618422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.618807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.717489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 862/1704] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:19.717489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd0d2d-0c30-461e-8f72-b8269b00e0fb None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.717489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd0d2d-0c30-461e-8f72-b8269b00e0fb None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.717489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd0d2d-0c30-461e-8f72-b8269b00e0fb None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.719932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 863/1705] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 24 07:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:19.724260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.725242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.725622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.752854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:19.755500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:19.772593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 843/1706] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:19.776550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:19.795204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] MFA Rules not processed for user `ea609a2504e64f6f9380061080976a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:19.799345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:19.805689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782cbff-caab-4c18-ad0b-6c7fd380c502 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:19.821085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:19.821280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.821563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.821844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.823642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:19.827188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7a4a-134f-4d65-90d3-888bb5ae9ab2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:19.887976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 844/1707] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:19 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:19.890855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:19.893300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:19.893583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:19.942244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:19.943567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:19.978920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 864/1708] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:19.979991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.007146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cbb77-9a71-4eba-b121-f0dfa6247261 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.080988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 845/1709] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:20.082296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.172608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:20.173568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.174195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.176547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.178426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.204495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.205989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0349b-44cd-4f9e-ac97-30906abe8dd4 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.228417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be7996-1525-422f-84cf-24455ce8bb9b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.228729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 865/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:20.233906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.234131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.234410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.247803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.248119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.248700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.248858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.249623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.264169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.292572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] MFA Rules not processed for user `ea609a2504e64f6f9380061080976a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:20.295676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:20.344246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 846/1711] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:20 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:20.344246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.350801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:20.379504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb97d-2a89-4b4b-80ec-2a831c3eeee5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.406849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.406849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.406849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.406849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.424430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.426508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.443486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 847/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:20.444592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.463705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:20.483348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 866/1713] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:20.484334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.507000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d48af-2e99-42a8-bd7c-8c5bd97a1a71 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.524351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.525077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.529596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.529982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.530786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.533779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.585329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 867/1714] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:20.586382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.592719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00e3db-2aaa-4fb1-a8d8-1fec840d3516 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.597318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:20.602255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76672e39ff4e4842b392af17746a18dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a365ee7bbed4acd9a72c6eb2e49e246', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.617135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.617537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.618224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.618538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:20.623048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.624355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.643108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ed92-3d61-4585-be0e-e722d50457fc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.654048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 848/1715] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:20.655235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.660813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.661624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.662264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.662679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] PATH_INFO: `/v3/projects/8d4da50a4bc4464d90be0ca6bb0ca8f5` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.665538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4da50a4bc4464d90be0ca6bb0ca8f5)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.667779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.677146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fe30-1f9f-4ebe-a33e-e25176681fcf tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.693895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:20.694793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:20.695214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:20.695637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] PATH_INFO: `/v3/projects/3cfce6e54b76418482001769d2705eed/users/9f6fc3e068084bc1a88b7b46b8aaaecb/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:20.723237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfce6e54b76418482001769d2705eed, user_id=9f6fc3e068084bc1a88b7b46b8aaaecb, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:20.726408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:20.746763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 849/1716] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:20 2022] PUT /identity/v3/projects/3cfce6e54b76418482001769d2705eed/users/9f6fc3e068084bc1a88b7b46b8aaaecb/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:20.747779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.804376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4da50a4bc4464d90be0ca6bb0ca8f5', 'request_id': 'req-26d0bb79-5e56-4daf-8045-d4fa88801f5b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:20.822978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4da50a4bc4464d90be0ca6bb0ca8f5', 'request_id': 'req-26d0bb79-5e56-4daf-8045-d4fa88801f5b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:20.834092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4da50a4bc4464d90be0ca6bb0ca8f5', 'request_id': 'req-26d0bb79-5e56-4daf-8045-d4fa88801f5b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:20.854756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] The token cache is being invalidate because project 8d4da50a4bc4464d90be0ca6bb0ca8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:20.856004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:20.856409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:20.863165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 868/1717] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:20 2022] DELETE /identity/v3/projects/8d4da50a4bc4464d90be0ca6bb0ca8f5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:20.864552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:20.869029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:20.897803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439d8-1958-4e17-8061-cfb710cc35d2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:20.975397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:21.040375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0bb79-5e56-4daf-8045-d4fa88801f5b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:21.052780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:21.055627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.058223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.058507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.069757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:21.071414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab460c7-bfbb-4231-8649-727616807b6d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:21.077982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76672e39ff4e4842b392af17746a18dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a365ee7bbed4acd9a72c6eb2e49e246', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:21.085871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.088750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.089191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.090334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:21.098327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:21.098919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 869/1718] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:21.100345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c476a-d49a-4934-952e-e700e17b06fa None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.100804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c476a-d49a-4934-952e-e700e17b06fa None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.101235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c476a-d49a-4934-952e-e700e17b06fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.148690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-133c476a-d49a-4934-952e-e700e17b06fa None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:21.151923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-133c476a-d49a-4934-952e-e700e17b06fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.193754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 870/1719] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:21.206582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.207095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.208877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.216108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 850/1720] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:20 2022] POST /identity/v3/projects => generated 354 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:21.218919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:21.335275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6a77a-6f76-4e7e-9029-88863e2f1974 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:21.342571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:21.343364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.343711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.344088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] PATH_INFO: `/v3/projects/3cfce6e54b76418482001769d2705eed/users/9f6fc3e068084bc1a88b7b46b8aaaecb/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.348911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfce6e54b76418482001769d2705eed, user_id=9f6fc3e068084bc1a88b7b46b8aaaecb, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:21.357021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:21.357425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.364503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5fbd1-471e-420f-b077-f62e69ae318c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:21.379783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 851/1721] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:21 2022] PUT /identity/v3/projects/3cfce6e54b76418482001769d2705eed/users/9f6fc3e068084bc1a88b7b46b8aaaecb/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:21.391893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.392892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.393171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.430941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:21.461414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:21.464631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.494987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 871/1722] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:21.498169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:21.504457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.615803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 852/1723] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:21.616851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:21.620236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.771709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:21.779871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27e9af-929f-45fe-b333-36b854a12aa2 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:21.845516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb47ddf-2aa5-463d-a3b6-5b03454c10d3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:21.854126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:21.855777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.856537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.857050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.860397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:21.926366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76672e39ff4e4842b392af17746a18dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a365ee7bbed4acd9a72c6eb2e49e246', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:21.927074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.927375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.927927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.930651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:21.931692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:21.936784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b197c-78a0-4161-821f-53b83cc8ef3f tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:21.972559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 872/1724] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:21.976737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a71a2-81b3-4611-8082-9a315d76316f None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.977493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a71a2-81b3-4611-8082-9a315d76316f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.977722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a71a2-81b3-4611-8082-9a315d76316f None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:21.978014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:21.980321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:21.987611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 873/1725] 10.222.0.42 () {54 vars in 961 bytes} [Wed Aug 24 07:40:21 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:21.989099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:21.994859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:21.995221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.018430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 853/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:22.019334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.041495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b62d6-90c9-4bf7-a001-bdb2df705944 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.048325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.049233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.049727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.050166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.052271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:22.053248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.055629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:22.075672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 854/1727] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:22.077260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.080202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.083856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] MFA Rules not processed for user `9f6fc3e068084bc1a88b7b46b8aaaecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:22.085082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.188695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:22.204952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 874/1728] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:22.206173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.207963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.254039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfd0a-bf58-4ddb-918f-b8d9a1e43712 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.267346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.268227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.268816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.269249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.326570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f851c-9360-4f66-961e-ff048eef53e1 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.335637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.336272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.336556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.336837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.338813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.352773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.360252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.361350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:22.362861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819f1c-aeac-4670-aac5-4d43d3c95054 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:22.367554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:22.376263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 855/1729] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:40:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:40:22.379988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.380281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.380587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.382840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 875/1730] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:22.385261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.413562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd503b7f-34e6-4dbf-91f1-0b85946a352a service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.427094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.428038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.428955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:22.432298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.433753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.434189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.439147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.442703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:22.509805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 856/1731] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Aug 24 07:40:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:22.512819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.580570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 876/1732] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:22 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:22.581599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:22.583954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e5b3-96e3-4407-affc-657b233040d1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.585097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:22.761497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa588d-6cf9-402c-bc5e-ef9d43ffb4bc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:22.785928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.786576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.786890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.787213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.848956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76672e39ff4e4842b392af17746a18dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a365ee7bbed4acd9a72c6eb2e49e246', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:22.850182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:22.850787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:22.851227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] PATH_INFO: `/v3/projects/2aad4674172141209339d2d6e2dddcc8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:22.853290 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2aad4674172141209339d2d6e2dddcc8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.855875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:22.996614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:22.996614 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:23.018618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 877/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:23.021997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.065397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aad4674172141209339d2d6e2dddcc8', 'request_id': 'req-8286d8ef-460f-45a4-9e90-795782796dd9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:23.093689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aad4674172141209339d2d6e2dddcc8', 'request_id': 'req-8286d8ef-460f-45a4-9e90-795782796dd9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:23.101881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:23.107350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aad4674172141209339d2d6e2dddcc8', 'request_id': 'req-8286d8ef-460f-45a4-9e90-795782796dd9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:23.149094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] The token cache is being invalidate because project 2aad4674172141209339d2d6e2dddcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:23.149396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:23.163731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:23.170990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 857/1734] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:22 2022] DELETE /identity/v3/projects/2aad4674172141209339d2d6e2dddcc8 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:23.172008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.177926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:23.285939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d307b3-bff9-4f21-a322-02d57cdd0810 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.519299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286d8ef-460f-45a4-9e90-795782796dd9 tempest-VolumeQuotaClassesTest-980886553 tempest-VolumeQuotaClassesTest-980886553-project-admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.537755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.538931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.539451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.539872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.542404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.542763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:23.543067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.543339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.543614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.544520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:23.546547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:23.566785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:23.571278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:23.617753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 858/1735] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:23 2022] POST /identity/v3/users => generated 293 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:23.618934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.630112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:23.649051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 878/1736] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:23.651245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.680554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:23.717935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4353d1f-f7dd-403f-b1cf-47cfef0626c2 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.740072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.740751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.740981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.741256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.754797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:23.768212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:23.810133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 879/1737] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:23.811076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.823071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc923-cfd9-4a42-811d-88f2c2136d4d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.840074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.842150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdf294-6644-487f-8037-e66d3e461942 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.842493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.842875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.843988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.847288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:23.854012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.873074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.873074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.873308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.884577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:23.890477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:23.914739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 880/1738] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:23.919234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:23.965155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a441f-ddeb-4b67-a591-82dadde05567 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:23.982479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:23.983505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:23.985469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:23.989660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] PATH_INFO: `/v3/projects/b60a1fbc56bd4e38a450362dcb72068c/users/95de2f30ce24450893b2e7a42661c072/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:23.993190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:23.994688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e3d6-8f93-4446-a564-6c7e6f52b651 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:24.012838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 859/1739] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:24.033659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60a1fbc56bd4e38a450362dcb72068c, user_id=95de2f30ce24450893b2e7a42661c072, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:24.040162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:24.090184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 881/1740] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:23 2022] PUT /identity/v3/projects/b60a1fbc56bd4e38a450362dcb72068c/users/95de2f30ce24450893b2e7a42661c072/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:24.093164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:24.093710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:24.094786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:24.176455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] MFA Rules not processed for user `95de2f30ce24450893b2e7a42661c072`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:24.182167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:24.412116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:24.414964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 860/1741] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:24.613241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ec09-d016-4eed-8f34-cf668a5c0b88 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:24.664382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:24.665059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:24.665314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:24.665592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:24.666236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:24.668208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:24.704591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:24.753922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:24.755574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:24.812137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00ffa1-0e50-43d8-ac44-dcb8e0f60e1e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:24.827928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 882/1742] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:24 2022] POST /identity/v3/projects => generated 387 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:24.833032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:24.844652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:24.847114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:24.849244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:24.850619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] PATH_INFO: `/v3/users/24241700e02c4a68ad5a36287c1986c3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:24.873042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24241700e02c4a68ad5a36287c1986c3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:24.883890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.044576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755f3a-0a31-49c3-b36b-56a3bbf05831 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.053111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.054154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.054265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.054589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.055608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.059599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af799d-c3c5-4873-9df3-379f62f51f39 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.112716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 883/1743] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:24 2022] POST /identity/v3/users => generated 300 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:25.119861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e937-c0bc-452e-baae-90b1a0aafc91 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.120271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e937-c0bc-452e-baae-90b1a0aafc91 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.121424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e937-c0bc-452e-baae-90b1a0aafc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.128687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24241700e02c4a68ad5a36287c1986c3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:25.129860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24241700e02c4a68ad5a36287c1986c3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:25.138091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:25.150024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24241700e02c4a68ad5a36287c1986c3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:25.151634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24241700e02c4a68ad5a36287c1986c3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:25.173341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24241700e02c4a68ad5a36287c1986c3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:25.209671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b5a0e937-c0bc-452e-baae-90b1a0aafc91 None None] MFA Rules not processed for user `ea609a2504e64f6f9380061080976a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:25.211494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b5a0e937-c0bc-452e-baae-90b1a0aafc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:25.215514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 861/1744] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:24 2022] DELETE /identity/v3/users/24241700e02c4a68ad5a36287c1986c3 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:25.215514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:25.216590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:25.348034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732b705-9cb3-4141-a260-7313ce88e7b9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.351955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 884/1745] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:25.353704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.354040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.354323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.356402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.357440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.357947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.358431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.359611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.364394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.386052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 862/1746] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:25.386414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:25.387993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:25.390391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:25.400673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 885/1747] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:25.404659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052587e0-3a50-4293-8add-eea49aec3f9f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.410479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.411146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:25.411736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.412017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.412289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] PATH_INFO: `/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9/users/51a5b269c12444aaa04535f5b1a8d494/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.414032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:25.433418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a502393a7d74c2a9cd7722ae9c7a6e9, user_id=51a5b269c12444aaa04535f5b1a8d494, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.437391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.468506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 863/1748] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:25 2022] PUT /identity/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9/users/51a5b269c12444aaa04535f5b1a8d494/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:25.473243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:25.517711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eebbbd-0e0b-4b60-bb6e-db3fd31fe2a6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.587179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:25.623001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade87e-95c3-4ab5-89f8-4f9b25c8dcdb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.630477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.631138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.631418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.633288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.633969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.635896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.645646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.646274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.646551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.646855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.658552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 864/1749] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:25.662891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:25.664407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.667838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10132b-7349-4b00-874a-c529153280c3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.683973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5c397-a542-4d61-83d5-abdc4f964303 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:25.691188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 886/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:25.708328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:25.709201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.709549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.709831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] PATH_INFO: `/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9/users/51a5b269c12444aaa04535f5b1a8d494/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.712152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a502393a7d74c2a9cd7722ae9c7a6e9, user_id=51a5b269c12444aaa04535f5b1a8d494, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:25.715440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:25.729112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 865/1751] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:25 2022] PUT /identity/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9/users/51a5b269c12444aaa04535f5b1a8d494/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:25.741620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:25.743294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:25.743698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:25.824283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] MFA Rules not processed for user `51a5b269c12444aaa04535f5b1a8d494`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:25.825577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:26.106348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 887/1752] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:26.143991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:26.146390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:26.232960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:26.368590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46b79-dff9-400c-9072-8ff6cbf7b185 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:26.390485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:26.391869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:26.392285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:26.392656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:26.434283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:26.435799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c5acc-d58a-4b96-81f6-e615473ce0b1 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:26.471166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 866/1753] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:26.548535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:26.791496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82896d95-25d3-41ff-a56f-43dad8731a50 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:26.802481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:26.806047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:26.809027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:26.809310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] PATH_INFO: `/v3/users/95de2f30ce24450893b2e7a42661c072` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:26.815114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=95de2f30ce24450893b2e7a42661c072)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:26.816991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:26.833502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79027395-d1d0-404c-8b69-28572116c8e4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:26.833786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79027395-d1d0-404c-8b69-28572116c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:26.834056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79027395-d1d0-404c-8b69-28572116c8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:26.928704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-79027395-d1d0-404c-8b69-28572116c8e4 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:26.928704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-79027395-d1d0-404c-8b69-28572116c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:27.006813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95de2f30ce24450893b2e7a42661c072'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.007574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95de2f30ce24450893b2e7a42661c072'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.008379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95de2f30ce24450893b2e7a42661c072'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.042531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:27.049457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95de2f30ce24450893b2e7a42661c072'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.065224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95de2f30ce24450893b2e7a42661c072'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.125487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 888/1754] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:26 2022] DELETE /identity/v3/users/95de2f30ce24450893b2e7a42661c072 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:27.126329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:27.127922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:27.134712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 867/1755] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:27.135835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:27.136122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:27.136389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:27.209998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:27.211403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:27.349856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb6d4-d14c-4fbe-a921-e3df4fb7bd3f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:27.363598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:27.365084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:27.369504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:27.373266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] PATH_INFO: `/v3/projects/b2aeb9902d8541cb97f42ddf9dd1089e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:27.377119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2aeb9902d8541cb97f42ddf9dd1089e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:27.383348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:27.445012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 868/1756] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:27.454595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:27.471614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:27.595982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2aeb9902d8541cb97f42ddf9dd1089e', 'request_id': 'req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.635179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2aeb9902d8541cb97f42ddf9dd1089e', 'request_id': 'req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.654636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2aeb9902d8541cb97f42ddf9dd1089e', 'request_id': 'req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.754706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] The token cache is being invalidate because project b2aeb9902d8541cb97f42ddf9dd1089e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:27.755708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.763231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:27.765385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 889/1757] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:27 2022] DELETE /identity/v3/projects/b2aeb9902d8541cb97f42ddf9dd1089e => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:27.766904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:27.779876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:27.838163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078425a6-8f30-44a2-b8a7-88ab0c201308 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:27.902036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:28.091384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:28.092391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:28.092744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:28.093088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:28.095140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:28.119077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8dc4-89a7-4950-abb9-5fd8de2e1fda admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:28.132172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:28.134548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:28.134894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:28.136263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:28.152630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:28.180161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:28.182613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:28.196543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 869/1758] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:40:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:40:28.197567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:28.198992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:28.250117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:28.288569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dca257-37c0-4187-a8b7-b15a7b0c88e1 service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:28.298420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:28.299281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:28.299565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:28.299836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] PATH_INFO: `/v3/users/76672e39ff4e4842b392af17746a18dd` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:28.301013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76672e39ff4e4842b392af17746a18dd)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:28.303351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:28.435352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76672e39ff4e4842b392af17746a18dd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:28.437052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76672e39ff4e4842b392af17746a18dd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:28.459328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76672e39ff4e4842b392af17746a18dd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:28.463636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76672e39ff4e4842b392af17746a18dd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:28.465881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:28.470570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:28.492910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:28.504532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-89ada06e-91ae-4224-95de-5dc90ee841e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76672e39ff4e4842b392af17746a18dd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:28.537126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 890/1759] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:28.540649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 870/1760] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:28 2022] DELETE /identity/v3/users/76672e39ff4e4842b392af17746a18dd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:29.623273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:29.626155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:29.674387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fba97-85f3-4431-b50d-dce34e400a7e service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:29.738517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:29.739552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:29.739910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:29.856425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:29.862826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.062171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:30.063904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.064195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.064470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] PATH_INFO: `/v3/users/024eff730dbd4df985368f24aaa157d4` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.065794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024eff730dbd4df985368f24aaa157d4)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:30.075337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:30.077119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 871/1761] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 24 07:40:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:30.155946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:30.159745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.204767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abb89b-112a-4162-8fa2-290b1fcca494 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:30.215467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024eff730dbd4df985368f24aaa157d4'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.216153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024eff730dbd4df985368f24aaa157d4'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.217256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024eff730dbd4df985368f24aaa157d4'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.230080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:30.231938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.231938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.231938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] PATH_INFO: `/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.233906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a365ee7bbed4acd9a72c6eb2e49e246)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:30.236381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:30.236761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024eff730dbd4df985368f24aaa157d4'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.253182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-936f2f70-389f-432a-b02a-4d16293dd628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024eff730dbd4df985368f24aaa157d4'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.279957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 891/1762] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:29 2022] DELETE /identity/v3/users/024eff730dbd4df985368f24aaa157d4 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:30.286630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.288198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.288663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.317702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:30.319175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.374085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:30.426365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a365ee7bbed4acd9a72c6eb2e49e246', 'request_id': 'req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.431636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 892/1763] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:30.432796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:30.434886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.448572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a365ee7bbed4acd9a72c6eb2e49e246', 'request_id': 'req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.463185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a365ee7bbed4acd9a72c6eb2e49e246', 'request_id': 'req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.498234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] The token cache is being invalidate because project 3a365ee7bbed4acd9a72c6eb2e49e246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:30.498588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.499749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:30.511972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 872/1764] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:30 2022] DELETE /identity/v3/projects/3a365ee7bbed4acd9a72c6eb2e49e246 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:30.513007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:30.521367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.540420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a88b0-19c6-4308-8d12-7c239125a45f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:30.620294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:30.621161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.621508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.622029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.624878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.643300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:30.646598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6fa69-251f-4873-9ccb-7fed2b8bb4e9 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:30.657847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 893/1765] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:30.660310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.660882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.661289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.696507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:30.698072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.730443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db12eaa-4b4d-4d5e-bd85-844c547fb6cc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:30.737060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:30.738077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.738843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.739432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.741500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.795173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:30.841900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 894/1766] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:30.841900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:30.841900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:30.868370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816498da-aa7d-49d5-873d-dd102d53c76e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:30.884214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:30.887236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.887955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.888825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] PATH_INFO: `/v3/users/88224875ce204bdd97ce6a20f0df8fa3` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.891896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88224875ce204bdd97ce6a20f0df8fa3)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:30.897978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:30.953860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:30.955738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf32d28-caa8-4296-9c40-c35170ddfc4a service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:30.972040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 873/1767] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:30.973549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96a68-a623-4013-8c28-8a82924073df None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:30.974244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96a68-a623-4013-8c28-8a82924073df None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:30.974560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96a68-a623-4013-8c28-8a82924073df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:30.985377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:31.003317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-01d96a68-a623-4013-8c28-8a82924073df None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:31.005075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-01d96a68-a623-4013-8c28-8a82924073df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:31.018856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 874/1768] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:31.019805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88224875ce204bdd97ce6a20f0df8fa3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:31.020914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88224875ce204bdd97ce6a20f0df8fa3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:31.021644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96a68-a623-4013-8c28-8a82924073df None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.022081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88224875ce204bdd97ce6a20f0df8fa3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:31.035995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96a68-a623-4013-8c28-8a82924073df None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:31.049591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:31.050228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.051170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88224875ce204bdd97ce6a20f0df8fa3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:31.053219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.054258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.055697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:31.055961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:31.068153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:31.085545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3626ebaf-811f-419d-b566-375e0e02d026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88224875ce204bdd97ce6a20f0df8fa3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:31.087622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 875/1769] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:40:31.097561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.106749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:31.117020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 895/1770] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:30 2022] DELETE /identity/v3/users/88224875ce204bdd97ce6a20f0df8fa3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:31.121687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.122058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.122319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.141886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced84a-efe0-4ef6-80f9-669d1851f5ab admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:31.171273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:31.176804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:31.293184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:31.295854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.296408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.298570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.299638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:31.303029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:31.341779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 896/1771] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:31.400031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 876/1772] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:31 2022] POST /identity/v3/projects => generated 391 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:31.404552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.440507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6853f1-ddab-4c38-9e01-1cdf71edc777 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:31.630743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.636262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:31.695029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:31.697950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.698247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.699751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.703718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:31.711340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:31.795067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 897/1773] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:31 2022] POST /identity/v3/users => generated 296 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:31.798007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.803210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:31.867911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda250-c930-4226-8e06-ad6f848f562e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:31.875838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:31.876438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.876900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.877177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] PATH_INFO: `/v3/projects/b60a1fbc56bd4e38a450362dcb72068c` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.878716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60a1fbc56bd4e38a450362dcb72068c)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:31.882281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:31.897298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b00697-4912-4606-9260-0f0cff85b406 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:31.910905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:31.912210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:31.912708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:31.913647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:31.943699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:31.945525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:31.971268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 877/1774] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:31.972485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:31.999013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08496d8-3cf1-4ebc-aa4c-71267796ffa3 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:32.005757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:32.048390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:32.049561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60a1fbc56bd4e38a450362dcb72068c', 'request_id': 'req-a3de1db6-1df7-44c9-892c-b58efdd90344'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.064303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60a1fbc56bd4e38a450362dcb72068c', 'request_id': 'req-a3de1db6-1df7-44c9-892c-b58efdd90344'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.074078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60a1fbc56bd4e38a450362dcb72068c', 'request_id': 'req-a3de1db6-1df7-44c9-892c-b58efdd90344'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.100056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] The token cache is being invalidate because project b60a1fbc56bd4e38a450362dcb72068c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:32.100350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.101105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.108837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 898/1775] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:31 2022] DELETE /identity/v3/projects/b60a1fbc56bd4e38a450362dcb72068c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:32.109939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:32.111615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:32.185829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:32.186841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.187348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.188536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.189531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:32.191462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:32.244377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 878/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:32.245357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:32.247210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:32.257147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1db6-1df7-44c9-892c-b58efdd90344 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:32.269395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b950d84-4647-4780-896d-99a820144c47 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:32.274569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:32.275311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.275586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.275860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] PATH_INFO: `/v3/users/51a5b269c12444aaa04535f5b1a8d494` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.276626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:32.277408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.277845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.277996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a5b269c12444aaa04535f5b1a8d494)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:32.278438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.279299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:32.281958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:32.285145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:32.302180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 879/1777] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:32.303964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:32.317725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c956705-fc85-4ba4-8709-e75c731e8c80 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:32.329988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:32.330882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.331318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.331753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] PATH_INFO: `/v3/projects/a4351fb5e7a54bf2a755f8039fc2cc8a/users/9f296054392b4267a4abbee51db7e673/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.406204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4351fb5e7a54bf2a755f8039fc2cc8a, user_id=9f296054392b4267a4abbee51db7e673, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:32.411227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e7130-ce77-4204-b703-c2c4db18da44 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:32.441555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a5b269c12444aaa04535f5b1a8d494'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.442809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a5b269c12444aaa04535f5b1a8d494'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.443517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a5b269c12444aaa04535f5b1a8d494'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.457406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 880/1778] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:32 2022] PUT /identity/v3/projects/a4351fb5e7a54bf2a755f8039fc2cc8a/users/9f296054392b4267a4abbee51db7e673/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:32.460926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.462354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.462821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.472691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a5b269c12444aaa04535f5b1a8d494'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.489241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a5b269c12444aaa04535f5b1a8d494'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:32.536063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] MFA Rules not processed for user `9f296054392b4267a4abbee51db7e673`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:32.539514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:32.560120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 899/1779] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:32 2022] DELETE /identity/v3/users/51a5b269c12444aaa04535f5b1a8d494 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:32.756791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 881/1780] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:32.761306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:32.763761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:32.910531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:32.970286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06e15-a365-47fa-9cd8-17677a2bb4e9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:32.975322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:32.976091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:32.976469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:32.976841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:32.977619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:32.979688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:33.017135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 900/1781] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:32 2022] POST /identity/v3/projects => generated 393 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:33.017472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.195367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34982e2b-563e-4fd6-961f-7e43de25b5a7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:33.220334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:33.303165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:33.303988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:33.304571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:33.304995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:33.306149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:33.327535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:33.411744 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.428733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 882/1782] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:33 2022] POST /identity/v3/users => generated 303 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:33.434224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beab5c3-9e6e-4dc3-a8b4-17ee00d7b52e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:33.447926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:33.448565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:33.448844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:33.449111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:33.455739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:33.458339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:33.519489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.538082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 901/1783] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:33.542134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318f40c-1a52-4e16-a034-f9d4be5bbe2c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:33.560160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:33.561371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:33.561814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:33.562103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] PATH_INFO: `/v3/projects/83f9b52da1024a4f81354866264f5a63/users/3c1aa385344044f7a63b4307169900ef/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:33.625659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f9b52da1024a4f81354866264f5a63, user_id=3c1aa385344044f7a63b4307169900ef, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:33.632565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:33.667534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 883/1784] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:33 2022] PUT /identity/v3/projects/83f9b52da1024a4f81354866264f5a63/users/3c1aa385344044f7a63b4307169900ef/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:33.671487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.692338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727f4c-dbde-4fd1-947e-677cb0150b50 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:33.906063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.906779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:33.907600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:33.907952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:33.908066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:33.908293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:33.911845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:33.911845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:33.955148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 902/1785] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:33.958239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:33.966332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:33.971295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860d6d1-409f-41a6-9c3e-88a8a64312b4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:33.995147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:33.995874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:33.996206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:33.996471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] PATH_INFO: `/v3/projects/f4d335b660ed43b099882d21b3c713c2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:33.997801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d335b660ed43b099882d21b3c713c2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:33.999977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:34.004300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7adb-2e75-47ab-bb37-9b6b05ff5ba2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:34.012709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:34.015343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.015999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.017453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] PATH_INFO: `/v3/projects/6677843a7adf46c09f48866baa371dda` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.019293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6677843a7adf46c09f48866baa371dda)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:34.021853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:34.185233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d335b660ed43b099882d21b3c713c2', 'request_id': 'req-b47c0ba8-d297-401e-b00c-3e8c9451751f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.205315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d335b660ed43b099882d21b3c713c2', 'request_id': 'req-b47c0ba8-d297-401e-b00c-3e8c9451751f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.207678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6677843a7adf46c09f48866baa371dda', 'request_id': 'req-ac6dece9-8789-4f17-97f7-ed599f9a89b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.239379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d335b660ed43b099882d21b3c713c2', 'request_id': 'req-b47c0ba8-d297-401e-b00c-3e8c9451751f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.246465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6677843a7adf46c09f48866baa371dda', 'request_id': 'req-ac6dece9-8789-4f17-97f7-ed599f9a89b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.267952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6677843a7adf46c09f48866baa371dda', 'request_id': 'req-ac6dece9-8789-4f17-97f7-ed599f9a89b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.277136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] The token cache is being invalidate because project f4d335b660ed43b099882d21b3c713c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:34.277429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.279959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.284110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 884/1786] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:33 2022] DELETE /identity/v3/projects/f4d335b660ed43b099882d21b3c713c2 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:34.287364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:34.292275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:34.299720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] The token cache is being invalidate because project 6677843a7adf46c09f48866baa371dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:34.299938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.301990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.309408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 903/1787] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:33 2022] DELETE /identity/v3/projects/6677843a7adf46c09f48866baa371dda => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:34.314024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:34.319008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:34.460903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0ba8-d297-401e-b00c-3e8c9451751f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:34.485403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:34.495501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:34.547164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6dece9-8789-4f17-97f7-ed599f9a89b3 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:34.561985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:34.561985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.561985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.561985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] PATH_INFO: `/v3/users/9f6fc3e068084bc1a88b7b46b8aaaecb` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.571176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6fc3e068084bc1a88b7b46b8aaaecb)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:34.573236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:34.585404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:34.587408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.587914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.588393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] PATH_INFO: `/v3/projects/83f9b52da1024a4f81354866264f5a63/users/3c1aa385344044f7a63b4307169900ef/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.593801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f9b52da1024a4f81354866264f5a63, user_id=3c1aa385344044f7a63b4307169900ef, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:34.599088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:34.615005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 885/1788] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:34 2022] PUT /identity/v3/projects/83f9b52da1024a4f81354866264f5a63/users/3c1aa385344044f7a63b4307169900ef/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:34.616861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:34.619209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:34.672233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6fc3e068084bc1a88b7b46b8aaaecb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.677143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6fc3e068084bc1a88b7b46b8aaaecb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.679098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6fc3e068084bc1a88b7b46b8aaaecb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.707339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6fc3e068084bc1a88b7b46b8aaaecb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.724473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-760dc4d7-33c8-4a1f-b853-fc455644054a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6fc3e068084bc1a88b7b46b8aaaecb'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.767633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3b38d-ab82-4ebf-8d8f-982b267134ef admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:34.772150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 904/1789] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:34 2022] DELETE /identity/v3/users/9f6fc3e068084bc1a88b7b46b8aaaecb => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:34.773551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca26b-21fc-46d9-9ead-59684c53c14b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.773839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca26b-21fc-46d9-9ead-59684c53c14b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.774119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca26b-21fc-46d9-9ead-59684c53c14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.784026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:34.784501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.784786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.785063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] PATH_INFO: `/v3/users/be2bf90b865e4787854626088188c234` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.788577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be2bf90b865e4787854626088188c234)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:34.790737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:34.841131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-f29ca26b-21fc-46d9-9ead-59684c53c14b None None] MFA Rules not processed for user `3c1aa385344044f7a63b4307169900ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:34.843315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f29ca26b-21fc-46d9-9ead-59684c53c14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:34.897402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be2bf90b865e4787854626088188c234'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.898101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be2bf90b865e4787854626088188c234'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.912010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be2bf90b865e4787854626088188c234'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.913276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be2bf90b865e4787854626088188c234'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.957732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03ea38ac-9cc8-4e12-b9ef-fbe3a0581f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be2bf90b865e4787854626088188c234'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:34.985948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 886/1790] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:34 2022] DELETE /identity/v3/users/be2bf90b865e4787854626088188c234 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:34.987975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:34.988261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:34.988541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:34.994287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 887/1791] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:40:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:34.996351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:35.000356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.053738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 905/1792] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:35.066590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:35.070510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a28f-20b9-4308-aa2b-ca41ed8643fd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.071829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a28f-20b9-4308-aa2b-ca41ed8643fd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.072133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a28f-20b9-4308-aa2b-ca41ed8643fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.120651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24673e-6480-4ad9-87e8-4ca56ef43ed9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:35.131898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:35.132881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.133447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.134328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] PATH_INFO: `/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.135786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a502393a7d74c2a9cd7722ae9c7a6e9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:35.138377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:35.142962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7f14a28f-20b9-4308-aa2b-ca41ed8643fd None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:35.144489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7f14a28f-20b9-4308-aa2b-ca41ed8643fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.248187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a502393a7d74c2a9cd7722ae9c7a6e9', 'request_id': 'req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:35.268458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a502393a7d74c2a9cd7722ae9c7a6e9', 'request_id': 'req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:35.277280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a502393a7d74c2a9cd7722ae9c7a6e9', 'request_id': 'req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:35.288271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:35.306440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] The token cache is being invalidate because project 1a502393a7d74c2a9cd7722ae9c7a6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:35.306873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:35.307939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:35.311882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 888/1793] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:34 2022] DELETE /identity/v3/projects/1a502393a7d74c2a9cd7722ae9c7a6e9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:35.312573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:35.316449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.423397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 906/1794] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:35.431375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ca4b-c667-46df-bff8-0fa0908fe8fd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.432611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ca4b-c667-46df-bff8-0fa0908fe8fd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.434796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ca4b-c667-46df-bff8-0fa0908fe8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.452318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7df3-a3c9-4555-bfdf-97733044f3e5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:35.461021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:35.461759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.462080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.462395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.468757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.539640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e6e5ca4b-c667-46df-bff8-0fa0908fe8fd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:35.540865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e6e5ca4b-c667-46df-bff8-0fa0908fe8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.631562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:35.633585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:35.657959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 889/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:35 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:35.720298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:35.722198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.764502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 907/1796] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:35.771502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:35.775257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.775850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.776284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.823934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:35.825866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.847033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 908/1797] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:35.847996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:35.868638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ddbcb-a5d8-49e5-b274-7c3f96de705f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:35.881775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:35.882331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.882610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.882915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.886180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:35.936000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436c335-3a87-4766-bd6d-a7017780cb80 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:35.946998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:35.947906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:35.961845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:35.962563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:35.986138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:35.988246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.021108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 890/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:36.021952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.041233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.043059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.050400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbeabb-c396-4a2b-bd6b-4b507e58eb3e service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.065453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.066123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.066408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.066682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.067493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:36.067838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.069720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879617-0b1b-40df-9694-becb04416515 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.088858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 909/1799] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:36.097034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 891/1800] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:36.101633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.104829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcad9-640e-4252-858c-9c5b42c619c8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.105104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcad9-640e-4252-858c-9c5b42c619c8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.105386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcad9-640e-4252-858c-9c5b42c619c8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.107081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 892/1801] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:36.127664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bcc0b-503d-4156-b50c-07d7e982a2e0 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.134146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.134881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.135279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.135667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.136456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.138793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.159214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.159525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.159775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.196048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:36.197816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:36.212891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 893/1802] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:40:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:36.259291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 910/1803] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:36 2022] POST /identity/v3/projects => generated 389 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:36.266601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.285376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ae80-562d-474f-9831-856a7fa5cb17 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.481138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.485731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.486270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.486952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.487666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.489738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.559224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 894/1804] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:36 2022] POST /identity/v3/users => generated 295 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:36.560467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.569546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.571403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:36.599435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be0188-a050-4760-a1d5-021df1095360 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.619118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.621621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.621904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.622758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.623442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.630919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.688772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 911/1805] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:36.690094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.721343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d862-1e1b-48f9-94fb-e6fab911bb36 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.737888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.738580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.738890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.739164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.739825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.741452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.762564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 912/1806] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:36.770767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:36.796133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ce6c4-aafa-4415-afc3-26343dc210bc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.811641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:36.812588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.813590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.813945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] PATH_INFO: `/v3/projects/bad0daeb509941c5bdea47ddeb16714b/users/20285901caed41979814514ab859339e/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:36.852372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad0daeb509941c5bdea47ddeb16714b, user_id=20285901caed41979814514ab859339e, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:36.857940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744edbd1-9ecc-416d-b99e-83a44b6712fa admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:36.869592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:36.887213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:36.907160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 913/1807] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:36 2022] PUT /identity/v3/projects/bad0daeb509941c5bdea47ddeb16714b/users/20285901caed41979814514ab859339e/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:36.910099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ea43-6083-4a71-94c9-087cbe282a19 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:36.920538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:36.921389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:36.921898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.002037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] MFA Rules not processed for user `20285901caed41979814514ab859339e`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:37.005042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:37.189530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 914/1808] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:37.189530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:37.189530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:37.233172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:37.236107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:37.236638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:37.237112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.315989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:37.319805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:37.339139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 895/1809] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 772 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:37.341079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:37.382507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ff39-d2ea-4708-b854-7b6e922e0867 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:37.389466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:37.390739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:37.390988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:37.391784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] PATH_INFO: `/v3/projects/3cfce6e54b76418482001769d2705eed` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.393997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfce6e54b76418482001769d2705eed)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:37.398050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:37.517183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfce6e54b76418482001769d2705eed', 'request_id': 'req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:37.523823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8c2b-dda2-4bef-8fa5-49a8f496460a service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:37.545832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfce6e54b76418482001769d2705eed', 'request_id': 'req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:37.576463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfce6e54b76418482001769d2705eed', 'request_id': 'req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:37.622266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] The token cache is being invalidate because project 3cfce6e54b76418482001769d2705eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:37.622611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:37.624902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a611d28f-22f4-48c2-8d8a-99f8ddde0e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:37.635439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 915/1810] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:37 2022] DELETE /identity/v3/projects/3cfce6e54b76418482001769d2705eed => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:37.637018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7af35e-c3c2-45a0-bc63-d51fa383080e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:37.637526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7af35e-c3c2-45a0-bc63-d51fa383080e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:37.637725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7af35e-c3c2-45a0-bc63-d51fa383080e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.702193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ab7af35e-c3c2-45a0-bc63-d51fa383080e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:37.704725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ab7af35e-c3c2-45a0-bc63-d51fa383080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:37.716007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:37.717547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:37.720024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:37.721054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.733223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:37.742109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:37.751756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 916/1811] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:37.753653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:37.754013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:37.754406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:37.820776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:37.832230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:37.842232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 896/1812] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:37 2022] POST /identity/v3/projects => generated 387 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:37.843682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:37.847736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:37.855858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:38.027912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:38.222258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 917/1813] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:38.223298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.261834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea315be7-e0b0-49b2-932a-b188bf9baaab None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.274957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ee30d-ef47-4f58-baf9-686da8a0b5e9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.291274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.291915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.292248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.292525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.293326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:38.293484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.301931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.304983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.306488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.307705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.308974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.314574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.344103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.365805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 918/1814] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:38.368511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.376293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:38.410883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b79c-5a82-4f60-8ed7-8cded1449f92 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.422584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.424234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.425182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.426348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] PATH_INFO: `/v3/projects/767a52e90c134821857fdc7a0d5560e6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.431709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=767a52e90c134821857fdc7a0d5560e6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.436873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.444315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 897/1815] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:37 2022] POST /identity/v3/users => generated 300 bytes in 602 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:38.460731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.528170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493810f0-c988-4e7f-ba46-7dbbf5dce166 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.553708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.554347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.554627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.554930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.570396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.575478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.584609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767a52e90c134821857fdc7a0d5560e6', 'request_id': 'req-6cf26f17-a78a-4496-8d38-2728582bfcf6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:38.602558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767a52e90c134821857fdc7a0d5560e6', 'request_id': 'req-6cf26f17-a78a-4496-8d38-2728582bfcf6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:38.619713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767a52e90c134821857fdc7a0d5560e6', 'request_id': 'req-6cf26f17-a78a-4496-8d38-2728582bfcf6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:38.653131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 898/1816] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:38 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:38.654624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.659348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] The token cache is being invalidate because project 767a52e90c134821857fdc7a0d5560e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:38.663623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:38.667907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:38.679754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 919/1817] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:38 2022] DELETE /identity/v3/projects/767a52e90c134821857fdc7a0d5560e6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:38.680662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.686124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:38.755640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b875e2-7d32-491d-9320-1f602c82203b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.761958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.762598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.763200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.763495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.764165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.768283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb763b0-d076-4688-ae9d-4ca8de3f2304 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.783339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 899/1818] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:38.784957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.785248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.785528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.816247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:38.817489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:38.825925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf26f17-a78a-4496-8d38-2728582bfcf6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.828172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 900/1819] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:38.829974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:38.834936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:38.840272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.840941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.841238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.841579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.842282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.849190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f0732-af5b-4385-b6df-5947a2a5c94b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.879048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084067b-8013-4288-a5c8-d266b6aa9ff6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:38.891670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:38.893261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.893479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.895220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 920/1820] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:38 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:38.896681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:38.898057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] PATH_INFO: `/v3/projects/b79804f0741148fe8bd5236f44db19a8/users/e4fb12dd37d14188b88e4d6fe9208722/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.898985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:38.899295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:38.954805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79804f0741148fe8bd5236f44db19a8, user_id=e4fb12dd37d14188b88e4d6fe9208722, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:38.960460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:38.979637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:38.985939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:39.001671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 901/1821] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:38 2022] PUT /identity/v3/projects/b79804f0741148fe8bd5236f44db19a8/users/e4fb12dd37d14188b88e4d6fe9208722/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:39.002853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:39.005390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.160595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 921/1822] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:39.172561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.216710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125299-dbcc-4999-807b-52dcd165b072 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.225684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.227891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.228220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.228507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.229201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.231207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.241655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:39.266843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 922/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:39.266843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.271137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9ae5e-2126-4850-aa9c-648d3edbdd16 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.286239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.287828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.289184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.291034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.291957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.294213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ff3d1-9e29-48e8-915e-d42dfa887292 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.295706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.305579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.308545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.317124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.318070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.320799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:39.322197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.325897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.331341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 902/1824] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:39.332561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.358880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 923/1825] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:39.361086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757edd6-5e55-49ff-9728-c101f651b333 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.362651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.373945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.376124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.376396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.376675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] PATH_INFO: `/v3/projects/b79804f0741148fe8bd5236f44db19a8/users/e4fb12dd37d14188b88e4d6fe9208722/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.382488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79804f0741148fe8bd5236f44db19a8, user_id=e4fb12dd37d14188b88e4d6fe9208722, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.386471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ce6e-dc3d-460b-a9d1-3e44841dbbfc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.387960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.413285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.413928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.414468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.420290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.422674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.427060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81063a7-4d4e-4cfb-9a0f-23b88694c53b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.435677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 903/1826] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:39 2022] PUT /identity/v3/projects/b79804f0741148fe8bd5236f44db19a8/users/e4fb12dd37d14188b88e4d6fe9208722/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:39.447016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.483348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 924/1827] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:39.488269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.488567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.491266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.560128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] MFA Rules not processed for user `e4fb12dd37d14188b88e4d6fe9208722`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:39.563697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:39.644869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a514d-c62d-4f85-a7e7-27564c10cb8a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:39.683825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:39.684536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:39.684994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:39.685366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:39.686297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:39.707441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:39.816126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 925/1828] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:39.817178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.829724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 904/1829] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:39 2022] POST /identity/v3/projects => generated 363 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:39.835476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:39.837689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:39.872068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e7cd3-9ad5-4e4a-9645-b3bfd5f42626 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.004062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae40d1-6fe5-4125-b11a-a15dcd019ec7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.009583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.016982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.017581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.018004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.088892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.089580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.089858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.096611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] PATH_INFO: `/v3/projects/2cccac621d2c479a84f2093a797261e8/users/9abaa3a77e2f41e6905a88315aa0b5ba/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.104101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.106477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.108790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:40.131820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 905/1830] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:40.132980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.157023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc1451-28f6-460a-a388-a2448a9760d3 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.167498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.168171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.168454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.168731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.169373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.172926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.185565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cccac621d2c479a84f2093a797261e8, user_id=9abaa3a77e2f41e6905a88315aa0b5ba, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.189378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3223ce-0d62-4371-ae12-6ee546069f40 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.201233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:40.234337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 906/1831] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:40 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:40.239008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.267064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de183e13-1e2d-4bee-9c9e-61a219b6ad12 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.286785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 926/1832] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:39 2022] PUT /identity/v3/projects/2cccac621d2c479a84f2093a797261e8/users/9abaa3a77e2f41e6905a88315aa0b5ba/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:40.310992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.311337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.311573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.372140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] MFA Rules not processed for user `9abaa3a77e2f41e6905a88315aa0b5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:40.377425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:40.452201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.455229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.455798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.456375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.460367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.466721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.495493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 907/1833] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:40.497989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.525088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bda9c0-f7b5-451a-b131-c03fb3773334 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.536311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.536936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.537230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.537666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.538572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.540373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.541519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 927/1834] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:40.544030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.558759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eb3c7-d66c-4c61-940c-00f07541e490 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.590734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.591677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.592174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.592685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.593870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.598034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.614740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 908/1835] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:40.615743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.649750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 928/1836] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:40 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:40.651619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.651740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117ca4f-86c6-4848-aa6c-64875393c905 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.741646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.742509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.743575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.743575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] PATH_INFO: `/v3/users/20285901caed41979814514ab859339e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.744347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20285901caed41979814514ab859339e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.746415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.788179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21162f9-5a87-4ceb-a6e8-e3241a62d61a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:40.800020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:40.801016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.801303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.801585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] PATH_INFO: `/v3/projects/1e8d07e64117439a8d3d821d4f1d8674/users/58036ba6b96e406b81d2387ffe1aad1f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.812790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20285901caed41979814514ab859339e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:40.813915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20285901caed41979814514ab859339e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:40.824465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20285901caed41979814514ab859339e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:40.825467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20285901caed41979814514ab859339e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:40.825935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8d07e64117439a8d3d821d4f1d8674, user_id=58036ba6b96e406b81d2387ffe1aad1f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:40.830215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:40.843741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34dd9eb5-383b-4d11-9ca6-d07945404027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20285901caed41979814514ab859339e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:40.862258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 929/1837] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:40 2022] PUT /identity/v3/projects/1e8d07e64117439a8d3d821d4f1d8674/users/58036ba6b96e406b81d2387ffe1aad1f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:40.863689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:40.869267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:40.881085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:40.887424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 909/1838] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:40 2022] DELETE /identity/v3/users/20285901caed41979814514ab859339e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:40.890232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a41d1-5ea2-483f-8dfa-2680c134da93 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:40.894890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a41d1-5ea2-483f-8dfa-2680c134da93 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:40.895717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a41d1-5ea2-483f-8dfa-2680c134da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:40.921512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:40.941939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-354a41d1-5ea2-483f-8dfa-2680c134da93 None None] MFA Rules not processed for user `58036ba6b96e406b81d2387ffe1aad1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:40.944164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-354a41d1-5ea2-483f-8dfa-2680c134da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:40.964357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724b52e-769f-46f9-bea1-f8253923fbfd admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.049621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.050440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.050910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.051659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.052446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.055485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.061818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 910/1839] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:41.065633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.065633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.065633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.104271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 930/1840] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:40 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:41.105169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.107091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.107476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:41.108753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.128075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5a199-6a7c-4cc7-92a3-35504a11a0ba admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.129718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 911/1841] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:41.130560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.133146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.134000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.134291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.134568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.135266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.137201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.146661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96a6fe-5f99-4a35-9c5a-22cf729b8985 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.152682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.153339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.153649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.153928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.154578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.156245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.181739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 912/1842] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:41.181739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.182030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 931/1843] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:41 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:41.183000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.183185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.245056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621283cd-6703-41cb-bafb-f9b0b1be2535 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.250808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6b377-dc1e-4f6a-8019-09456ddd8649 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.256165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.256834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.257158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.257376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] PATH_INFO: `/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a/users/a7d47c52153747f299b7760aeaf4dd93/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.275123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=893d3ad2f6cc46bebb594fb8bbe2057a, user_id=a7d47c52153747f299b7760aeaf4dd93, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.277471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.296043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 932/1844] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:41 2022] PUT /identity/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a/users/a7d47c52153747f299b7760aeaf4dd93/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:41.296900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.303330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.303940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.304397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.304671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.333499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.336364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.354388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:41.361415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 913/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:41.362492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.374665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc78f-ee76-49bc-ae82-c2831b346067 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.425866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:41.436073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216daf0-2aa5-43f7-8f33-67891e33f8e5 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.452330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.453004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.453394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.454515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.455210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.457149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.467476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.468096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.468518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.468962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.470626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.474666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.490451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 914/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:41.495277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.506071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 933/1847] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:41 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:41.514960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.517830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730bb4-1cc1-437f-b527-62c65ebec832 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.523428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.524167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.524531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.524770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] PATH_INFO: `/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a/users/a7d47c52153747f299b7760aeaf4dd93/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.527203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=893d3ad2f6cc46bebb594fb8bbe2057a, user_id=a7d47c52153747f299b7760aeaf4dd93, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.527347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac12e-71cd-4619-b61f-bf4ed5e9d3ae admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.530140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.533382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.534346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.534756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.535175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.535976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.537683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5b15-fa0f-4792-9fe7-408b3d7a0bc9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.546127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 915/1848] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:41 2022] PUT /identity/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a/users/a7d47c52153747f299b7760aeaf4dd93/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:41.554596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 934/1849] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:41.559620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.562536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.563798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.564070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.638374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] MFA Rules not processed for user `a7d47c52153747f299b7760aeaf4dd93`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:41.639754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.655615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a213e4b-4d4b-447e-99f6-6c0a9ab08760 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.661437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.662197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.662509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.662811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] PATH_INFO: `/v3/projects/514c2a267e264dbebd496224afa523d6/users/8e95071a4fd14dbf9934b35ea6f92838/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.678652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=514c2a267e264dbebd496224afa523d6, user_id=8e95071a4fd14dbf9934b35ea6f92838, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.681474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.701293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 916/1850] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:41 2022] PUT /identity/v3/projects/514c2a267e264dbebd496224afa523d6/users/8e95071a4fd14dbf9934b35ea6f92838/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:41.702629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.705467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.740330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 935/1851] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:41.741188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.757416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b41d71-5add-4320-8698-626d12d6c710 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.788005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc55cc-66f7-4953-b611-336c8b5958ff admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.793392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.794031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.794307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.794579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] PATH_INFO: `/v3/users/9f296054392b4267a4abbee51db7e673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.795730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f296054392b4267a4abbee51db7e673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.798145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.840943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.841489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.841773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.842063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.842719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.843763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:41.844901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.850783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f296054392b4267a4abbee51db7e673'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:41.851754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f296054392b4267a4abbee51db7e673'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:41.855591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 936/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:41.856427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.858084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.861552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f296054392b4267a4abbee51db7e673'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:41.862356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f296054392b4267a4abbee51db7e673'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:41.874957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f296054392b4267a4abbee51db7e673'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:41.890505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 917/1853] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:41 2022] DELETE /identity/v3/users/9f296054392b4267a4abbee51db7e673 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:41.895182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:41.896596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:41.898119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:41.925810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf33ac-2675-46d2-a00a-61fd0d0b9fa6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.968758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8334ff-17ab-4bd3-b3ef-f8d1febef5ed admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:41.974044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.974680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.974990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.975265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] PATH_INFO: `/v3/projects/bad0daeb509941c5bdea47ddeb16714b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.976378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad0daeb509941c5bdea47ddeb16714b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:41.977624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:41.978235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:41.978415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:41.978552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:41.978805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:41.980887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.033700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad0daeb509941c5bdea47ddeb16714b', 'request_id': 'req-5728230e-e3b0-4efe-b61e-f7765008e0cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.041394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad0daeb509941c5bdea47ddeb16714b', 'request_id': 'req-5728230e-e3b0-4efe-b61e-f7765008e0cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.047964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad0daeb509941c5bdea47ddeb16714b', 'request_id': 'req-5728230e-e3b0-4efe-b61e-f7765008e0cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.054168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.055857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75578bc8-e948-412a-8f64-87fc8b45a90e service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.059407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] The token cache is being invalidate because project bad0daeb509941c5bdea47ddeb16714b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:42.059703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.060419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.062277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 918/1854] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:41 2022] DELETE /identity/v3/projects/bad0daeb509941c5bdea47ddeb16714b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:42.063321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.064638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.069906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 937/1855] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:42.071207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.071496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.071778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.091777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:42.094197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.123562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:42.148718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728230e-e3b0-4efe-b61e-f7765008e0cd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.157553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.158341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.158744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.159118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] PATH_INFO: `/v3/projects/514c2a267e264dbebd496224afa523d6/users/8e95071a4fd14dbf9934b35ea6f92838/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.162031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=514c2a267e264dbebd496224afa523d6, user_id=8e95071a4fd14dbf9934b35ea6f92838, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.166511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.200248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 919/1856] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:42 2022] PUT /identity/v3/projects/514c2a267e264dbebd496224afa523d6/users/8e95071a4fd14dbf9934b35ea6f92838/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:42.212296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.214164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.217063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 938/1857] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:42.218230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.219923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.234286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dd671-faba-4c6b-b25a-98357fcdb2d0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.279623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:42.309737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a129ef-08d8-46bd-a278-14b3ec4a3477 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.315692 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.316656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.317088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.317427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.318200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.318312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] PATH_INFO: `/v3/users/e4fb12dd37d14188b88e4d6fe9208722` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.318473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.318789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] PATH_INFO: `/v3/users/9abaa3a77e2f41e6905a88315aa0b5ba` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.319986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abaa3a77e2f41e6905a88315aa0b5ba)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.320330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fb12dd37d14188b88e4d6fe9208722)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.322112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.322794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.390471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fb12dd37d14188b88e4d6fe9208722'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.391223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fb12dd37d14188b88e4d6fe9208722'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.397377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abaa3a77e2f41e6905a88315aa0b5ba'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.399100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abaa3a77e2f41e6905a88315aa0b5ba'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.400076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fb12dd37d14188b88e4d6fe9208722'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.400719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abaa3a77e2f41e6905a88315aa0b5ba'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.402293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fb12dd37d14188b88e4d6fe9208722'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.414685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abaa3a77e2f41e6905a88315aa0b5ba'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.422898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c95434b-eed3-42fd-a6ec-99b918551c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abaa3a77e2f41e6905a88315aa0b5ba'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.431948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fb12dd37d14188b88e4d6fe9208722'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:42.435996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 939/1858] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:42 2022] DELETE /identity/v3/users/9abaa3a77e2f41e6905a88315aa0b5ba => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:42.438584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff65c1-b4d2-41df-a351-fb0bb3440fcf None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.439010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff65c1-b4d2-41df-a351-fb0bb3440fcf None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.440030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff65c1-b4d2-41df-a351-fb0bb3440fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.449989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 920/1859] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:42 2022] DELETE /identity/v3/users/e4fb12dd37d14188b88e4d6fe9208722 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:42.451002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.453058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.491415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-52ff65c1-b4d2-41df-a351-fb0bb3440fcf None None] MFA Rules not processed for user `8e95071a4fd14dbf9934b35ea6f92838`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:42.492639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-52ff65c1-b4d2-41df-a351-fb0bb3440fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.545368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8baf5-b090-48c4-980f-ee4809f0c6c9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.551947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.553040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.553333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.553605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.555619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.595313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:42.617139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 940/1860] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:42.619323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.619993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.620742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.656311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:42.658457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.683833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.685501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.709377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 921/1861] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:42.710492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.714014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.725693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:42.781346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 941/1862] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:42.790986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2d57-e7fa-41f5-92f1-443f11686173 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.794871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.795577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.795913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.796254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.810981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.812699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.836136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 922/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:42.839197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.841178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:42.858877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5c652-7d71-4991-9fe9-de2ef71c516d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.884111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.884752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.885380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.885657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.914867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:42.916807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:42.930043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:42.939005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 942/1864] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:42.941560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9c3a1-fc8d-4ad3-9a04-c64adc560b03 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:42.948018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:42.951703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:42.951988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:42.952263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:42.954349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.028108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.029584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c9c6-ab44-48c1-b26b-24a9e2e8a9fe service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.040492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 923/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:43.235126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.236590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.249834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f725-13c2-48dc-9a50-3ab632fd3f8c service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:43.255489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:43.256182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.256462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.256737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] PATH_INFO: `/v3/projects/b79804f0741148fe8bd5236f44db19a8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.258244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79804f0741148fe8bd5236f44db19a8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.260219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.326464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79804f0741148fe8bd5236f44db19a8', 'request_id': 'req-857d9310-337d-4283-9692-6d6ee31f6bb8'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.336481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79804f0741148fe8bd5236f44db19a8', 'request_id': 'req-857d9310-337d-4283-9692-6d6ee31f6bb8'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.344858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79804f0741148fe8bd5236f44db19a8', 'request_id': 'req-857d9310-337d-4283-9692-6d6ee31f6bb8'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.357918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] The token cache is being invalidate because project b79804f0741148fe8bd5236f44db19a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:43.358210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.358957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-857d9310-337d-4283-9692-6d6ee31f6bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.360850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 943/1866] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:43 2022] DELETE /identity/v3/projects/b79804f0741148fe8bd5236f44db19a8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:43.365960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.366244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.366512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.387872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:43.388712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.483507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:43.510776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 924/1867] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:43.518259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.518549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.518862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.532801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.537231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.572809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:43.574336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.581049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bdfdb-71dd-4bab-b5b0-51f9b2f4dae1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:43.597718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:43.598734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.598982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 944/1868] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:43.599433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.599722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] PATH_INFO: `/v3/projects/2cccac621d2c479a84f2093a797261e8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.602633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.603727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cccac621d2c479a84f2093a797261e8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.606344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.620312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05f7b-85c8-41b1-847f-fe3ea3a31674 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:43.665430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:43.666205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.667060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.669161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.669921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:43.670326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.673841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.691145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 945/1869] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:43.691145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.691145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.700679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cccac621d2c479a84f2093a797261e8', 'request_id': 'req-6c9351ee-b446-478c-9efc-7ccca2dede5b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.705062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8827-a67d-4b67-bdda-59eefc3bef19 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:43.710494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cccac621d2c479a84f2093a797261e8', 'request_id': 'req-6c9351ee-b446-478c-9efc-7ccca2dede5b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.720110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cccac621d2c479a84f2093a797261e8', 'request_id': 'req-6c9351ee-b446-478c-9efc-7ccca2dede5b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.735744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] The token cache is being invalidate because project 2cccac621d2c479a84f2093a797261e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:43.736159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.737230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.739330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 925/1870] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:43 2022] DELETE /identity/v3/projects/2cccac621d2c479a84f2093a797261e8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:43.740567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.742964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.795141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:43.796726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.797015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.797345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] PATH_INFO: `/v3/projects/a4351fb5e7a54bf2a755f8039fc2cc8a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.798929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4351fb5e7a54bf2a755f8039fc2cc8a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.800895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.856060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:43.872794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9351ee-b446-478c-9efc-7ccca2dede5b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:43.873688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4351fb5e7a54bf2a755f8039fc2cc8a', 'request_id': 'req-252eed0d-89ba-4bd8-865f-0698428f1793'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.912104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4351fb5e7a54bf2a755f8039fc2cc8a', 'request_id': 'req-252eed0d-89ba-4bd8-865f-0698428f1793'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.923565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4351fb5e7a54bf2a755f8039fc2cc8a', 'request_id': 'req-252eed0d-89ba-4bd8-865f-0698428f1793'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.938913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] The token cache is being invalidate because project a4351fb5e7a54bf2a755f8039fc2cc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:43.939413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.940237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:43.942268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 946/1871] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:43 2022] DELETE /identity/v3/projects/a4351fb5e7a54bf2a755f8039fc2cc8a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:43.947118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:43.949085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:43.956063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:43.957028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:43.957244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:43.957608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:43.958328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:43.960616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:43.982210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eed0d-89ba-4bd8-865f-0698428f1793 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.007267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.010269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.010749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.011249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] PATH_INFO: `/v3/users/a7d47c52153747f299b7760aeaf4dd93` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.014873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d47c52153747f299b7760aeaf4dd93)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.019087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.019946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 926/1872] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:43 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:44.021388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:44.024441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:44.036590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:44.111268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d47c52153747f299b7760aeaf4dd93'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.112336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d47c52153747f299b7760aeaf4dd93'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.113196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d47c52153747f299b7760aeaf4dd93'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.130264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d47c52153747f299b7760aeaf4dd93'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.139590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d47c52153747f299b7760aeaf4dd93'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.141447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf4a5b-bae5-4263-b434-42100b4bd7dc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.148449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.149388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.149923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.150297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] PATH_INFO: `/v3/users/3c1aa385344044f7a63b4307169900ef` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.152094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1aa385344044f7a63b4307169900ef)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.154413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.158803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 947/1873] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:43 2022] DELETE /identity/v3/users/a7d47c52153747f299b7760aeaf4dd93 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:44.160178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:44.162808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:44.255682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1aa385344044f7a63b4307169900ef'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.259009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1aa385344044f7a63b4307169900ef'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.261544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74384a-7188-4390-8e41-6fbe3cdf6759 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.269542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1aa385344044f7a63b4307169900ef'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.271381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1aa385344044f7a63b4307169900ef'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.274865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.276221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.277844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.279149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.281130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.288919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.303122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1aa385344044f7a63b4307169900ef'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:44.355552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 948/1874] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:44 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:44.357963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:44.359408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 927/1875] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:44 2022] DELETE /identity/v3/users/3c1aa385344044f7a63b4307169900ef => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:44.369868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:44.391773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b63055-23d2-4f8a-9bab-ee7e2f346986 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.476579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=15,keystone:SELECT=80,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:44.574659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.575619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.575956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.576236 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.577022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.579184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.600878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 949/1876] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:44.604894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:44.634855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e74a-9ca7-4bab-8ca4-50396a323ee5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.660449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.661300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.663779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.664301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.665267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.672039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15d73-3270-4432-a91e-b7b8708a36d0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.750128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 928/1877] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:44.754875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:44.776832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209664-0d96-4ecb-8456-f09e1cd655c2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:44.786546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:44.787265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.787549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.787829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] PATH_INFO: `/v3/projects/bf4e57c56a044416ab03ad1dca739370/users/9d95eecc7778474fa0a1f49344bbc68f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.815008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4e57c56a044416ab03ad1dca739370, user_id=9d95eecc7778474fa0a1f49344bbc68f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:44.818122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:44.882608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 950/1878] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:44 2022] PUT /identity/v3/projects/bf4e57c56a044416ab03ad1dca739370/users/9d95eecc7778474fa0a1f49344bbc68f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:44.884213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:44.884729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:44.884997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:44.989371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] MFA Rules not processed for user `9d95eecc7778474fa0a1f49344bbc68f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:44.990623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.045685 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.047750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.192310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 929/1879] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:45.193284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.195665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.224329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:45.229987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4921085-1ff8-48ca-9064-53ca1c526ee6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.232651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8220-0d3a-4998-9700-9a5944adca05 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.241547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.242910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.243149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.243904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] PATH_INFO: `/v3/projects/83f9b52da1024a4f81354866264f5a63` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.247312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f9b52da1024a4f81354866264f5a63)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.248683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.249540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.252990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.254552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.255183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] PATH_INFO: `/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.262125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=893d3ad2f6cc46bebb594fb8bbe2057a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.271476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.367280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f9b52da1024a4f81354866264f5a63', 'request_id': 'req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.401371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f9b52da1024a4f81354866264f5a63', 'request_id': 'req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.412058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f9b52da1024a4f81354866264f5a63', 'request_id': 'req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.415835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '893d3ad2f6cc46bebb594fb8bbe2057a', 'request_id': 'req-98304e16-9194-4973-9688-534b76b2e615'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.427584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '893d3ad2f6cc46bebb594fb8bbe2057a', 'request_id': 'req-98304e16-9194-4973-9688-534b76b2e615'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.447411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '893d3ad2f6cc46bebb594fb8bbe2057a', 'request_id': 'req-98304e16-9194-4973-9688-534b76b2e615'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.447620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] The token cache is being invalidate because project 83f9b52da1024a4f81354866264f5a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:45.447729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.451154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.454340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 930/1880] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:45 2022] DELETE /identity/v3/projects/83f9b52da1024a4f81354866264f5a63 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:45.455148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.463131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] The token cache is being invalidate because project 893d3ad2f6cc46bebb594fb8bbe2057a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:45.463423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.464099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98304e16-9194-4973-9688-534b76b2e615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:45.469763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 951/1881] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:45 2022] DELETE /identity/v3/projects/893d3ad2f6cc46bebb594fb8bbe2057a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:45.471288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f97c2-7ada-4b4f-915e-75c0965eff83 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.471660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f97c2-7ada-4b4f-915e-75c0965eff83 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.471775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.471946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f97c2-7ada-4b4f-915e-75c0965eff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.495450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-cf3f97c2-7ada-4b4f-915e-75c0965eff83 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:45.496696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-cf3f97c2-7ada-4b4f-915e-75c0965eff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.528550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:45.592519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998a34-d6ea-4c34-b08d-1d21551ed0d1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.599564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.600922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.601210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.601489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.601757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 952/1882] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:45.603197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.603622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.604063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.611384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.615105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e698d3-3c77-4178-88c5-1728a27c0174 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.642348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:45.643607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.647358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 931/1883] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:45.648819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc7a-cf32-4569-aac8-148fb5e32073 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.649194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc7a-cf32-4569-aac8-148fb5e32073 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.649641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc7a-cf32-4569-aac8-148fb5e32073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.658854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 953/1884] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:45.659868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.661367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.674433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d864e-50c9-45d0-b37e-e9f37a558c40 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.688899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9151bc7a-cf32-4569-aac8-148fb5e32073 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:45.690321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9151bc7a-cf32-4569-aac8-148fb5e32073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.736404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.737040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.737376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.737634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.738294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.740282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.755340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 932/1885] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:45.757478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.757882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.758220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.765774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 954/1886] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:45.765774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.776081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde795a5-492e-4418-855d-798dc42ab786 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.778658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:45.779904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:45.781093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.781752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.782030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.782304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.782995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:45.783299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.784899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.788007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 933/1887] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:45.788795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.798784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 955/1888] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:45.799638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.804542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee8d6-4e4b-47eb-b798-0306cd3b4549 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.809450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.810061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.810334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.810604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.811322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.812138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dcc7e-0988-4a4e-b6cd-e3f57b527663 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.814081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.815634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:45.817971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.818650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.819153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.819442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.820224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:45.820703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.822534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.833076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 934/1889] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:45.833894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.844485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba01da0-c580-4372-bdac-694744208a1d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.848206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 956/1890] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:45.849045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.850781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.851734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.852238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.852723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.853551 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.855895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.859855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b461-ebaf-4baf-ac6e-6bdf22038806 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.866847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.867437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.867703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.867986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] PATH_INFO: `/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121/users/04a2812ccf274766adeedd3450da0dde/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.884282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319b6cf4ddde4dc9a719b2820fb6d121, user_id=04a2812ccf274766adeedd3450da0dde, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.885260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 935/1891] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:45.886085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.887108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.900704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 957/1892] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:45 2022] PUT /identity/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121/users/04a2812ccf274766adeedd3450da0dde/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:45.901495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:45.949395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe6c93-bc0a-4812-b862-8505202ef6ab admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.965189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131cfea-522e-4d35-b967-7f508d810362 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:45.969846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:45.970454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:45.970756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:45.971033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:45.971765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:45.973697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:45.981567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:45.999022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 958/1893] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:45.999022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.007055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b155-65b0-4958-9ede-f286383d9ed6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.008751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.009364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.009641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.010054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.010672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.011867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.012452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.012707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.012806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.012993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.013627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.015529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.026253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 959/1894] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:46.027130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.038457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505644-0228-4e87-9078-f3680073c92e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.040248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 936/1895] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:45 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:46.041039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.104910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.105521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.105802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.106076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.106739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.108342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.117203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fc3d-172d-4eac-b6d0-5d8aa73c9144 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.118717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 960/1896] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:46.119504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.122627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.123317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.123600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.123873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] PATH_INFO: `/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121/users/04a2812ccf274766adeedd3450da0dde/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.126553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] RBAC: Authorizing `identity:create_grant(project_id=319b6cf4ddde4dc9a719b2820fb6d121, user_id=04a2812ccf274766adeedd3450da0dde, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.129308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.129692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614b1a1-bbcf-476b-8baa-d6417538038d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.134484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.135098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.135382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.135658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.136078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:46.136277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.138455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff0a89-08a1-4eee-8ad6-be6ab94a079f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.141875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 937/1897] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:46 2022] PUT /identity/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121/users/04a2812ccf274766adeedd3450da0dde/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:46.143210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.169404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 961/1898] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:46 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:46.169404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.169404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.169404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.200666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] MFA Rules not processed for user `04a2812ccf274766adeedd3450da0dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:46.201907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:46.210349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d61f6-f6d3-47e2-87b8-dd548483046b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.215964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.216575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.216859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.217133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.217784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.220027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.230034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 938/1899] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:46.230034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.247614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526215-1776-48ac-ba8b-5125365c8dec admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.253415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.253966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.254242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.254513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.255215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.256813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.267906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 939/1900] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:46.268727 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.278366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25163b6c-cc36-45ff-a5e5-6b6daf8db0c6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.283470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 962/1901] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:46.283663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.284363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.284639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.284909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] PATH_INFO: `/v3/projects/5ee514090a934301b57adaf62d8b9c7e/users/4b633efed130495ca2c1c312aab133ce/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.285126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.298103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb276d-a78f-4c97-b01c-abad3896b6ff None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.305346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.306298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.306582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.307177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.308049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.308888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee514090a934301b57adaf62d8b9c7e, user_id=4b633efed130495ca2c1c312aab133ce, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.310505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.311411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.325457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 963/1902] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:46.325457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.327216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:46.330933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 940/1903] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:46 2022] PUT /identity/v3/projects/5ee514090a934301b57adaf62d8b9c7e/users/4b633efed130495ca2c1c312aab133ce/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:46.331737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.391681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:46.412806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaa786-ba42-4f3a-aa45-292a58e9eb03 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.418526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.419641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.420017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.420382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] PATH_INFO: `/v3/projects/4e32cf82cb92435f900fd6f1baf25faa/users/cb0c9d05f53345878f134eea1f61dd50/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.442140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e32cf82cb92435f900fd6f1baf25faa, user_id=cb0c9d05f53345878f134eea1f61dd50, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.445733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3d397-f2ed-41e2-b144-9636d8f02a36 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.463806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1278b5c-0f7c-46da-8e81-c8f852484be6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:46.465140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 941/1904] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:46 2022] PUT /identity/v3/projects/4e32cf82cb92435f900fd6f1baf25faa/users/cb0c9d05f53345878f134eea1f61dd50/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:46.466861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19189cf2-9e4b-49b2-8d88-82e0fd10f1c5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.467109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19189cf2-9e4b-49b2-8d88-82e0fd10f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.467508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19189cf2-9e4b-49b2-8d88-82e0fd10f1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.510350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-19189cf2-9e4b-49b2-8d88-82e0fd10f1c5 None None] MFA Rules not processed for user `4b633efed130495ca2c1c312aab133ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:46.514540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-19189cf2-9e4b-49b2-8d88-82e0fd10f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:46.598826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:46.604158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.604681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.605328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.653798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:46.743120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 942/1905] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:46.743120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc0fa-3bbe-4be5-b150-e9fd70ae7375 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.743120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc0fa-3bbe-4be5-b150-e9fd70ae7375 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.743120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc0fa-3bbe-4be5-b150-e9fd70ae7375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.787371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f95bc0fa-3bbe-4be5-b150-e9fd70ae7375 None None] MFA Rules not processed for user `cb0c9d05f53345878f134eea1f61dd50`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:46.789013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f95bc0fa-3bbe-4be5-b150-e9fd70ae7375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:46.804801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:46.806546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:46.819602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 964/1906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:46.820686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:46.822363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:46.973457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 943/1907] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:46.974747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6951f-9006-433f-8d80-5557be881c04 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:46.975038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6951f-9006-433f-8d80-5557be881c04 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:46.975309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6951f-9006-433f-8d80-5557be881c04 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:46.976956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 944/1908] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:46.977835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6951f-9006-433f-8d80-5557be881c04 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.015310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d570-ec5b-4678-bcdf-34ec1ddda0f6 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.021420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.022300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.022603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.023126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] PATH_INFO: `/v3/users/58036ba6b96e406b81d2387ffe1aad1f` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.024933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58036ba6b96e406b81d2387ffe1aad1f)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.027224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.071790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6951f-9006-433f-8d80-5557be881c04 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.080720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.080720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.080720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.080720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.081016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.082124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.109137 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:47.120904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58036ba6b96e406b81d2387ffe1aad1f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.123115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58036ba6b96e406b81d2387ffe1aad1f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.125241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58036ba6b96e406b81d2387ffe1aad1f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.140552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 945/1909] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:46 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:47.141811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.144416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.169968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58036ba6b96e406b81d2387ffe1aad1f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.183221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-88e3db54-3f21-4347-92eb-72dfea837224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58036ba6b96e406b81d2387ffe1aad1f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.214911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 965/1910] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:46 2022] DELETE /identity/v3/users/58036ba6b96e406b81d2387ffe1aad1f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:47.216269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.216537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.220638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.234289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:47.253468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:47.255197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.260321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7baeab-b821-46b5-b0a2-6cb49541cdc8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.365359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.365983 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.366263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.366542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.367207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.369512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53fa6-b18c-4c1a-9595-20b60097a171 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.370035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 966/1911] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:47.371090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.372712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.394077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865195c1-3236-4b9c-b5e7-de0e124068aa None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.400379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.401269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.401639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.402003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.409152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.411162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.415272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 946/1912] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/projects => generated 387 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:47.415272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.415272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.417800 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.438885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 967/1913] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/users => generated 296 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:47.440974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.443089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.452482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:47.454945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.482831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9722bfe-ce68-4c2a-a32e-d7c7db7228b3 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.546660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 947/1914] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:47.548549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.566566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16899-e125-490d-925d-2a5fdeafcf20 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.570523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.571615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.572046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.572640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.573422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.574503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.575918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.576312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.577666 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.579853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.584486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.586077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.598347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 968/1915] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:47.599234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.606660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 948/1916] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/users => generated 294 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:47.606660 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.608239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.614112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7424-e557-49ce-af42-e59912c5212b service nova] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.619380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.619987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.620263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.620531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.621171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.622866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01658f-8537-4c73-9877-db515386e712 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.622994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.628623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.629288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.629907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.629907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] PATH_INFO: `/v3/users/9d95eecc7778474fa0a1f49344bbc68f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.630999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d95eecc7778474fa0a1f49344bbc68f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.632939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.642117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 969/1917] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:47.642117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.642117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.648906 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:47.711116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d95eecc7778474fa0a1f49344bbc68f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.712499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d95eecc7778474fa0a1f49344bbc68f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.716212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e3af1-b12c-4bde-b4f3-3d1f48048a80 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.720251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.720872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.721200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.721448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d95eecc7778474fa0a1f49344bbc68f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.721577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.722442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d95eecc7778474fa0a1f49344bbc68f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.723299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.726132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:47.738316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.739885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.742772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d95eecc7778474fa0a1f49344bbc68f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:47.751601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 970/1918] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:47.752494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.753904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.761493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 949/1919] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:47 2022] DELETE /identity/v3/users/9d95eecc7778474fa0a1f49344bbc68f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:47.762566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.764274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.774662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cade81-a7f4-4b48-8ba1-0f5de12a51dc service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.885385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.886178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.886594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.886989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.887742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.889613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60fe5-9e42-4635-86ba-d9602979987f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.906180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 971/1920] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:47.908407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.908792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.909445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.915470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae8b8-652f-4aa0-bb58-b77278ce37e2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.937135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.938359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.938645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.938956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] PATH_INFO: `/v3/projects/0dfdd65649994f1e9e745403d140170e/users/093189908ac34c1d83b52fd74382ee07/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.940314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:47.942640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:47.958299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 972/1921] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:47.959341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:47.961887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfdd65649994f1e9e745403d140170e, user_id=093189908ac34c1d83b52fd74382ee07, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.965200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.972890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060c10-eb20-446d-b122-5b4de30b9094 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:47.980269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:47.981180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:47.981717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:47.982188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:47.983073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:47.985653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:47.998550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 973/1922] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:47.999709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.004492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.015875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 950/1923] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:47 2022] PUT /identity/v3/projects/0dfdd65649994f1e9e745403d140170e/users/093189908ac34c1d83b52fd74382ee07/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:48.015875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.139551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3274-5125-4b10-a142-9f94d7fbd6e8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.146398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290d22-7b87-41ab-9887-857656faeb83 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.152665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.153719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.154179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.154537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] PATH_INFO: `/v3/projects/6f4d803a439344d18e41661a9c8988e2/users/1cce4bb0eed249969d75b5b2e6e15b3a/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.179776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4d803a439344d18e41661a9c8988e2, user_id=1cce4bb0eed249969d75b5b2e6e15b3a, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.182498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.210753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 951/1924] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:48 2022] PUT /identity/v3/projects/6f4d803a439344d18e41661a9c8988e2/users/1cce4bb0eed249969d75b5b2e6e15b3a/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:48.212070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.236623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:48.263784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.264420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.264699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.267751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.285217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:48.297788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.300231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1043b8-8063-45cc-9a31-8fc87637e52d service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.316613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 974/1925] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:48.318321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.318755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.319225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.326904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c0ecc-86b8-4aeb-a28e-97723e040519 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.333935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.334687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.334994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.335263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.335969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.337593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.355229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 952/1926] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:48.357059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.360405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] MFA Rules not processed for user `1cce4bb0eed249969d75b5b2e6e15b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:48.361796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.376514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e087c7-87b0-4638-89a5-926ddbdade1f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.383364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.384045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.384321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.384635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] PATH_INFO: `/v3/projects/0dfdd65649994f1e9e745403d140170e/users/093189908ac34c1d83b52fd74382ee07/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.387102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfdd65649994f1e9e745403d140170e, user_id=093189908ac34c1d83b52fd74382ee07, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.392362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d0d3b-4a61-48d0-94f2-932f3c21b042 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.421881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 953/1927] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:48 2022] PUT /identity/v3/projects/0dfdd65649994f1e9e745403d140170e/users/093189908ac34c1d83b52fd74382ee07/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:48.424689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.425068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.425344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.455071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 975/1928] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:48.457268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.479012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] MFA Rules not processed for user `093189908ac34c1d83b52fd74382ee07`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:48.479435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ac28-cae9-4889-94e0-5a1939d988ca None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.481375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.639560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.641939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.642193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.642882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.643832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.646148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.679283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 954/1929] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:48.680289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.681671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.688208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 976/1930] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:48 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:48.689316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.696801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1c866-9ebe-4338-8dcc-d06999f178c1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.784011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46302f8-6ba0-4144-8780-d6968e9ac518 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.792175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.794066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.794188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.795036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.795414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.795782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] PATH_INFO: `/v3/projects/1e8d07e64117439a8d3d821d4f1d8674` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.796095 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.796599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.797830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.799837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.802585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8d07e64117439a8d3d821d4f1d8674)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:48.805163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:48.817648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:48.840894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 977/1931] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:48 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:48.841982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.845714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.902632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8d07e64117439a8d3d821d4f1d8674', 'request_id': 'req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:48.910092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:48.931857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2bbb-2a3d-4a96-a100-f8068cb53e0e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:48.940467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8d07e64117439a8d3d821d4f1d8674', 'request_id': 'req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:48.950764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8d07e64117439a8d3d821d4f1d8674', 'request_id': 'req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:48.965340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] The token cache is being invalidate because project 1e8d07e64117439a8d3d821d4f1d8674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:48.965902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:48.967169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:48.968962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 955/1932] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:48 2022] DELETE /identity/v3/projects/1e8d07e64117439a8d3d821d4f1d8674 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:48.970788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:48.972238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:48.992763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:48.993477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:48.993821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:48.994162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:48.995974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.066838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f950e2c-d3fc-404a-b25f-efafb2df0ae7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.072480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.073329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.073761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.074642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] PATH_INFO: `/v3/projects/bf4e57c56a044416ab03ad1dca739370` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.076856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4e57c56a044416ab03ad1dca739370)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.080829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.081014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.082385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.096694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 978/1933] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:49.100213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.101696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.118954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4ee84-46c4-44b1-aaa5-5703be66fc0c service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.127653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.128797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.129131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.129516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.130255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.134708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.147803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 979/1934] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:49.152677 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.154909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.163033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4e57c56a044416ab03ad1dca739370', 'request_id': 'req-84f7c6aa-04bf-41a6-b756-1141902b0c54'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.175597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8d54-2845-43da-b51a-c066684802b4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.175851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4e57c56a044416ab03ad1dca739370', 'request_id': 'req-84f7c6aa-04bf-41a6-b756-1141902b0c54'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.184576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4e57c56a044416ab03ad1dca739370', 'request_id': 'req-84f7c6aa-04bf-41a6-b756-1141902b0c54'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.228149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] The token cache is being invalidate because project bf4e57c56a044416ab03ad1dca739370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:49.228515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.234477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.236248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 956/1935] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:48 2022] DELETE /identity/v3/projects/bf4e57c56a044416ab03ad1dca739370 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:49.237441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.248165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:49.248458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.307968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.308853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.309141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.309461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] PATH_INFO: `/v3/users/8e95071a4fd14dbf9934b35ea6f92838` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.311238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e95071a4fd14dbf9934b35ea6f92838)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.313334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.370337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c6aa-04bf-41a6-b756-1141902b0c54 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.375681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.376018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.376350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.376627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] PATH_INFO: `/v3/projects/ee727636cce944b5894a8aafe96dc08d/users/abd26a3371eb46e0acdc19fa2ad69af0/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.398313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee727636cce944b5894a8aafe96dc08d, user_id=abd26a3371eb46e0acdc19fa2ad69af0, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.401798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.404579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:49.424430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 957/1936] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:49 2022] PUT /identity/v3/projects/ee727636cce944b5894a8aafe96dc08d/users/abd26a3371eb46e0acdc19fa2ad69af0/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:49.425454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.427315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.454796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e95071a4fd14dbf9934b35ea6f92838'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.455506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e95071a4fd14dbf9934b35ea6f92838'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.456147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e95071a4fd14dbf9934b35ea6f92838'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.474911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e95071a4fd14dbf9934b35ea6f92838'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.481600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e95071a4fd14dbf9934b35ea6f92838'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.499954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 980/1937] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:49 2022] DELETE /identity/v3/users/8e95071a4fd14dbf9934b35ea6f92838 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:49.499954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.499954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.504670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c60a-acb7-472a-a038-cb22ea659952 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.577793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456eafa-30e3-430c-b681-da611d034baa admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.587493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.588143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.588379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.588656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.592844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.606956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.607601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.607882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.608158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] PATH_INFO: `/v3/users/04a2812ccf274766adeedd3450da0dde` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.609291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a2812ccf274766adeedd3450da0dde)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.611382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.682441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a2812ccf274766adeedd3450da0dde'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.683073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a2812ccf274766adeedd3450da0dde'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.696114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.697941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.698508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a2812ccf274766adeedd3450da0dde'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.699318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a2812ccf274766adeedd3450da0dde'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.710913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 981/1938] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:49.711805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.713187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.719223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a2812ccf274766adeedd3450da0dde'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:49.729763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35acc01-ef64-447f-9195-7b4d7fb75fcc service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.736328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.736949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.737273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.737551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.738201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.740778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.744982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 958/1939] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:49 2022] DELETE /identity/v3/users/04a2812ccf274766adeedd3450da0dde => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:49.758148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 982/1940] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:49.759426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.761618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.796676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:49.901716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee2b27-1693-4709-9f80-5b0c896eb2b0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.909945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:49.910786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.911677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.911967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] PATH_INFO: `/v3/projects/ee727636cce944b5894a8aafe96dc08d/users/abd26a3371eb46e0acdc19fa2ad69af0/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:49.920859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee727636cce944b5894a8aafe96dc08d, user_id=abd26a3371eb46e0acdc19fa2ad69af0, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:49.924726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:49.926603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04134d-ee82-49bd-9666-8b0c1125ba27 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:49.926959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:49.958399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 959/1941] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:49 2022] PUT /identity/v3/projects/ee727636cce944b5894a8aafe96dc08d/users/abd26a3371eb46e0acdc19fa2ad69af0/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:49.962777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c2d0-a9e4-4421-8de4-a69b60fed99e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:49.966987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:49.969921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:49.970167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:49.970465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.017309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] MFA Rules not processed for user `abd26a3371eb46e0acdc19fa2ad69af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:50.018579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.100998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.101867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.102212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.114789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] PATH_INFO: `/v3/projects/514c2a267e264dbebd496224afa523d6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.118505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=514c2a267e264dbebd496224afa523d6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.126545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.171861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 960/1942] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:50.183831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:50.185431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.218409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514c2a267e264dbebd496224afa523d6', 'request_id': 'req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.226546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514c2a267e264dbebd496224afa523d6', 'request_id': 'req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.248388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514c2a267e264dbebd496224afa523d6', 'request_id': 'req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.289365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] The token cache is being invalidate because project 514c2a267e264dbebd496224afa523d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:50.291634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.292836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4003b5ee-fc57-45cc-b672-a15d6cf0ec74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.300093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 983/1943] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:49 2022] DELETE /identity/v3/projects/514c2a267e264dbebd496224afa523d6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:50.307225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.307510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.307859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.338430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251162c-84dd-4093-8e8e-feca44f80a46 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:50.363521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:50.369335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.516863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.517498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.518142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.518142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.523074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.535081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 984/1944] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:50.536196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:50.541105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.545684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:50.575839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419773-c9db-491c-b4e5-2c17b58d6e4a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:50.595856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.596419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.596819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.597083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] PATH_INFO: `/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.598509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=319b6cf4ddde4dc9a719b2820fb6d121)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.600267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.651084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.655868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925dbba-8e55-48fd-ab80-50bafadfcaf7 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.688160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 961/1945] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:50.690603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29a40-32a1-4f57-83b0-3edad64df77f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.691278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:50.693161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29a40-32a1-4f57-83b0-3edad64df77f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.694392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29a40-32a1-4f57-83b0-3edad64df77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.723089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319b6cf4ddde4dc9a719b2820fb6d121', 'request_id': 'req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.733041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3fc29a40-32a1-4f57-83b0-3edad64df77f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:50.734336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3fc29a40-32a1-4f57-83b0-3edad64df77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.768147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319b6cf4ddde4dc9a719b2820fb6d121', 'request_id': 'req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.777716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319b6cf4ddde4dc9a719b2820fb6d121', 'request_id': 'req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.801274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] The token cache is being invalidate because project 319b6cf4ddde4dc9a719b2820fb6d121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:50.801690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.802603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ab19289c-ab49-484b-9fdf-1d9cbb81e82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:50.805397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 985/1946] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:50 2022] DELETE /identity/v3/projects/319b6cf4ddde4dc9a719b2820fb6d121 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:50.809964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.810350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.810954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.846615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:50.847924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.880871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 986/1947] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:50.880871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:50.881184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 962/1948] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:50.884298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.884593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.884874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.889514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d434d-0ccd-4ab3-96d7-d75fc3423d9d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:50.903861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.905179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.905918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.911014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.912182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:50.917745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.917745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.918049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:50.918049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:50.931580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 987/1949] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:50.932717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:50.934707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 963/1950] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:50.944171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:50.957067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccb2e-a744-4628-8a70-889f94fe364e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:50.959008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36464-7adc-4e42-85aa-a98e1a179ba3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:50.961962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.962651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.963038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.963378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.964077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.966068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.969054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:50.969893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:50.970306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:50.970597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:50.971271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:50.971712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:50.973268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:50.985980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 964/1951] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:50.990750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.003072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a49b6-df06-4ee8-8069-2cd805b9922e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.006684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 988/1952] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/projects => generated 407 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:51.008282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.008887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.009164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.009437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.010680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.010969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.013463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.047915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 965/1953] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:50 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:51.052487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.102182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8aab2-c201-49d2-b169-76c3f1278419 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.161233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e802-abe2-4167-872f-67f76fc8ab41 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.168475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.169328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.169886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.170278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.171014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.173631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.183094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.184360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.185160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.186067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.188821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:51.189694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.194411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.211791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 966/1954] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:40:51.216444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.229794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3eb5a-0599-4e48-8063-d29532faf364 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.235593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.236364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.236762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.237138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.237889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.242056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 989/1955] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/users => generated 304 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:51.242234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.243254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.253417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 967/1956] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:51.257098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01780-b880-45e9-80e9-9fc2cf0d365b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.259146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.260017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.268967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.268967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.268967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.268967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.268967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.276247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fc345-9aad-466c-b624-0bc1c0471f5c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.279065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 990/1957] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:51.283820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.284836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.284953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.285031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.285281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.285965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.288015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.293036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:51.298931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df3983-0ea4-4911-b5ae-ed2ea2fd7932 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.306191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.306853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.307135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.307620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.319951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 968/1958] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:51.320294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.320392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.323286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.341080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 991/1959] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:51.341976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36423-3d5f-4783-b5ef-18e86c9d4ef1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.342209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.348777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.349791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.349791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.350079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] PATH_INFO: `/v3/projects/df2beeefd80a4cbdbbccefd8ef99131a/users/dc673f738b90454e951c2dd4e37f5b47/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.356559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4fd5-4684-493c-99c7-6d50c0f70740 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.364533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.365539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.366091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.366797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] PATH_INFO: `/v3/projects/c9deb08d60024fdebf1af8998a2ecf19/users/f5c9b325a5604a6aa427059fdd7f14a2/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.373487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2beeefd80a4cbdbbccefd8ef99131a, user_id=dc673f738b90454e951c2dd4e37f5b47, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.380654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5fbae-04e4-4498-a753-58c4f9a03267 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.387023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9deb08d60024fdebf1af8998a2ecf19, user_id=f5c9b325a5604a6aa427059fdd7f14a2, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.391978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e32be9-557a-4827-bbd2-88e494a31cea admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.414549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 992/1960] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:51 2022] PUT /identity/v3/projects/c9deb08d60024fdebf1af8998a2ecf19/users/f5c9b325a5604a6aa427059fdd7f14a2/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:51.427294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.428664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.430261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.436788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 969/1961] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:51 2022] PUT /identity/v3/projects/df2beeefd80a4cbdbbccefd8ef99131a/users/dc673f738b90454e951c2dd4e37f5b47/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:51.447581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.449342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.450476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.522348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] MFA Rules not processed for user `f5c9b325a5604a6aa427059fdd7f14a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:51.523733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:51.526796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] MFA Rules not processed for user `dc673f738b90454e951c2dd4e37f5b47`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:51.529798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:51.618012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 993/1962] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:51.618989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.620496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:51.644634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 970/1963] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:51.651967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.731331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f7616-3446-461b-a626-f24edcd24a11 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.735784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.736417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.736837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.737056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.741430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:51.811030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a145-f2be-4482-9c78-6174e43ddbb1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.820523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:51.821168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:51.821452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:51.821728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:51.822476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.825024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.866769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 971/1964] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/projects => generated 409 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:51.867022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:51.867116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.867730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:51.868146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:51.903167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:51.903773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 994/1965] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:51.904875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:51.919379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3cce-0691-4d18-895a-efbc78398ad7 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:51.934014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:51.998005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7717c-0c08-4531-8c5f-2e716d0c7d36 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.003058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.003877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.004299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.004711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.039241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.039865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.040450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.040739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.041645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.043653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.096502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 995/1966] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:51 2022] POST /identity/v3/users => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:52.096746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.098044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.100595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:52.112616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 972/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:52.115068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5febf-b2c4-401b-b32c-69a88e68a1f4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.123187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.125325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.127093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.127388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.128385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.131461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.145619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 996/1968] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:52.149828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:52.186749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7f0cf-75be-4df0-aae7-6f265e81bb00 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.192634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.193425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.193790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.194120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] PATH_INFO: `/v3/projects/ad192f9f8903442f9e1b3284029916e8/users/7ee8e4cea79e430e907d74b4df3feec9/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.223807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad192f9f8903442f9e1b3284029916e8, user_id=7ee8e4cea79e430e907d74b4df3feec9, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.230476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.257560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 973/1969] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:52 2022] PUT /identity/v3/projects/ad192f9f8903442f9e1b3284029916e8/users/7ee8e4cea79e430e907d74b4df3feec9/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:52.262911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:52.292155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7054d4f-55ec-4cb7-9faf-2b0fc9243561 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.446568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.447364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.447750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.448262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.449064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.451513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d206dc3-7d14-4de4-afaf-794e76bc9649 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.466760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 997/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:52.471922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:52.487181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829c5f7-76ed-4d3d-9adf-f754fe9bf6e8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.499679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.500886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.501414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.502524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] PATH_INFO: `/v3/projects/ad192f9f8903442f9e1b3284029916e8/users/7ee8e4cea79e430e907d74b4df3feec9/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.508964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad192f9f8903442f9e1b3284029916e8, user_id=7ee8e4cea79e430e907d74b4df3feec9, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.512396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.539130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 974/1971] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:52 2022] PUT /identity/v3/projects/ad192f9f8903442f9e1b3284029916e8/users/7ee8e4cea79e430e907d74b4df3feec9/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:52.551465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.551778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.552063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.612998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] MFA Rules not processed for user `7ee8e4cea79e430e907d74b4df3feec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:52.614269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:52.731156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 998/1972] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:52 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:52.743044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:52.804642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2bc39-32f7-40b6-a0aa-87ccc7328920 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:52.808477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:52.809114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:52.809361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:52.809631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:52.821326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:52.823184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:52.833439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 975/1973] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:52 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:53.035183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:53.055964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6131f0-7279-4b00-b885-3ce5cca0003b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:53.064932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:53.065822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:53.066403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:53.067114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:53.085168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:53.177462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:53.179366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20986c7d-d105-49e3-baf0-a061447c6ca4 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:53.190193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 999/1974] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:53 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:53.742580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:53.744204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:53.800717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:53.824778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbf118-58f0-4e18-8de5-e20b5ebfe8b5 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:53.830276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:53.830973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:53.831285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:53.831562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] PATH_INFO: `/v3/users/4b633efed130495ca2c1c312aab133ce` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:53.832749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b633efed130495ca2c1c312aab133ce)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:53.835838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:53.892381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b633efed130495ca2c1c312aab133ce'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:53.893092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b633efed130495ca2c1c312aab133ce'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:53.902081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b633efed130495ca2c1c312aab133ce'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:53.903063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b633efed130495ca2c1c312aab133ce'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:53.920191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b633efed130495ca2c1c312aab133ce'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:53.938901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 976/1975] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:53 2022] DELETE /identity/v3/users/4b633efed130495ca2c1c312aab133ce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:53.943773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:53.944080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:53.944352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:53.967918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:53.970838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:54.058763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1000/1976] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:54.072641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:54.074252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:54.165825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403201a-46b4-44dd-b719-4b09d4ce2cf8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:54.169720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:54.170414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:54.170795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:54.171139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:54.182397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:54.184430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:54.199110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 977/1977] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:40:54.841361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:54.843134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:54.865038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77616c5f-8979-431f-81dc-4b13206cc1f6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:54.879842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:54.880553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:54.880788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:54.881054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] PATH_INFO: `/v3/projects/5ee514090a934301b57adaf62d8b9c7e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:54.882537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee514090a934301b57adaf62d8b9c7e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:54.884300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:54.996777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee514090a934301b57adaf62d8b9c7e', 'request_id': 'req-179609f3-77af-4175-bb8e-1b321e64d225'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.018900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee514090a934301b57adaf62d8b9c7e', 'request_id': 'req-179609f3-77af-4175-bb8e-1b321e64d225'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.041780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee514090a934301b57adaf62d8b9c7e', 'request_id': 'req-179609f3-77af-4175-bb8e-1b321e64d225'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.058681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] The token cache is being invalidate because project 5ee514090a934301b57adaf62d8b9c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:55.059373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.063498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.066194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1001/1978] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:54 2022] DELETE /identity/v3/projects/5ee514090a934301b57adaf62d8b9c7e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:55.071126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:55.082300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f2a4-c8a2-4c33-a987-eb4c63009f3a service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:55.169980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:55.170739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.171025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.171233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] PATH_INFO: `/v3/users/093189908ac34c1d83b52fd74382ee07` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.172994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] RBAC: Authorizing `identity:delete_user(user_id=093189908ac34c1d83b52fd74382ee07)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:55.175538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:55.197328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:55.268000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093189908ac34c1d83b52fd74382ee07'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.268668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093189908ac34c1d83b52fd74382ee07'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.277544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093189908ac34c1d83b52fd74382ee07'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.278233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093189908ac34c1d83b52fd74382ee07'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.290971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f3810b1f-a194-4807-be68-b769bc52809f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093189908ac34c1d83b52fd74382ee07'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.306026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 978/1979] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:55 2022] DELETE /identity/v3/users/093189908ac34c1d83b52fd74382ee07 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:55.606160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:55.607656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.622296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179609f3-77af-4175-bb8e-1b321e64d225 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:55.655470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:55.682643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:55.683304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.683582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.683852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] PATH_INFO: `/v3/projects/0dfdd65649994f1e9e745403d140170e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.684959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dfdd65649994f1e9e745403d140170e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:55.686909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:55.732679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dfdd65649994f1e9e745403d140170e', 'request_id': 'req-c7e7f8b5-505a-46a5-aec6-3eb912069917'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.740196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dfdd65649994f1e9e745403d140170e', 'request_id': 'req-c7e7f8b5-505a-46a5-aec6-3eb912069917'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.746326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dfdd65649994f1e9e745403d140170e', 'request_id': 'req-c7e7f8b5-505a-46a5-aec6-3eb912069917'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.756413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] The token cache is being invalidate because project 0dfdd65649994f1e9e745403d140170e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:55.756699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.757336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.758826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1002/1980] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:55 2022] DELETE /identity/v3/projects/0dfdd65649994f1e9e745403d140170e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:55.763697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ba813-4942-4405-83d3-45232630486f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.763986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ba813-4942-4405-83d3-45232630486f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.764268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ba813-4942-4405-83d3-45232630486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.784417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-585ba813-4942-4405-83d3-45232630486f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:55.785593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-585ba813-4942-4405-83d3-45232630486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.841212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:55.842636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.853203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 979/1981] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:55.854850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.855138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.855426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.856007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f8b5-505a-46a5-aec6-3eb912069917 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:55.860625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:55.861251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.861529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.861799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] PATH_INFO: `/v3/users/dc673f738b90454e951c2dd4e37f5b47` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.863034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc673f738b90454e951c2dd4e37f5b47)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:55.864983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:55.874438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:55.875702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.883876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 980/1982] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:55.889668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:55.899771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624605c-d78d-455a-b698-8b29e54cc7a9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:55.904450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:55.905047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.905468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.905595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.906237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:55.906588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:55.908718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:55.913952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc673f738b90454e951c2dd4e37f5b47'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.914623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc673f738b90454e951c2dd4e37f5b47'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.915281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc673f738b90454e951c2dd4e37f5b47'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.921311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 981/1983] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:40:55.925867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:55.927787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.931566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc673f738b90454e951c2dd4e37f5b47'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.940233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-939b417f-c63e-4111-945a-1e9910d9cef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc673f738b90454e951c2dd4e37f5b47'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:55.941180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff86c3b-d046-468c-aab6-03c6b37c6c9b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:55.946109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:55.946734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.947012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.947283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.947905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:55.949816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:55.956320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1003/1984] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:40:55 2022] DELETE /identity/v3/users/dc673f738b90454e951c2dd4e37f5b47 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:55.960998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:55.961333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:55.961562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:55.982033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:55.982813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 982/1985] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:55 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:40:55.983299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:55.986993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.015469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:56.032837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:56.045405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1004/1986] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:56.051724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931123f8-3bf9-4744-b015-2213b864af0a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.055419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.057110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.057265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.058064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.058345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.058753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.059470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.061887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.088808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 983/1987] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:55 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:40:56.091673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.104503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9862ff-63e7-4078-933a-d35147a9ca9e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.109201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.109941 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.110321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.110733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.111484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.113260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.123862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 984/1988] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:56.124255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671260dd-7646-4fab-85e3-37ac981a1c78 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.127657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.128336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.129023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.129391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.129846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.138353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0d928-d043-4dd4-a796-55563ad82fae admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.141696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.143369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.145057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.145940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.146460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.146975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.148075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.150735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6acd6a-cab1-44fb-bd2c-d4da24b2dad3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.155219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1005/1989] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:56.163506 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 985/1990] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:56.167758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.178342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917be1-10e4-4c49-becf-1804e911abb2 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.184721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.185394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.185672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.185945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] PATH_INFO: `/v3/projects/1fd21d3eafe145049162fdee48446f2d/users/8c1d30d457d74c4e870eba4e2eb9384f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.202878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd21d3eafe145049162fdee48446f2d, user_id=8c1d30d457d74c4e870eba4e2eb9384f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.206264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.224926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1006/1991] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:56 2022] PUT /identity/v3/projects/1fd21d3eafe145049162fdee48446f2d/users/8c1d30d457d74c4e870eba4e2eb9384f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:40:56.230536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.231070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.231865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.268750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:56.271400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.363694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 986/1992] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:40:56.376230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.378983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.468414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3040b51-8106-4de7-b3f3-eef0aa769403 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.472220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.473247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.473539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.474790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.490857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.492515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753ff71-3513-4a0b-9963-f444ede304c6 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.503840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1007/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:56.571859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.573676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.600126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad42eb9-c0ed-4690-883b-b908fba3dee9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.607306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.608092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.608473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.608845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] PATH_INFO: `/v3/projects/df2beeefd80a4cbdbbccefd8ef99131a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.610646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2beeefd80a4cbdbbccefd8ef99131a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.612869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.699460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2beeefd80a4cbdbbccefd8ef99131a', 'request_id': 'req-f1353a06-8656-4b15-bce5-94d310b1eed3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:56.719447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2beeefd80a4cbdbbccefd8ef99131a', 'request_id': 'req-f1353a06-8656-4b15-bce5-94d310b1eed3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:56.726271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2beeefd80a4cbdbbccefd8ef99131a', 'request_id': 'req-f1353a06-8656-4b15-bce5-94d310b1eed3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:56.739028 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] The token cache is being invalidate because project df2beeefd80a4cbdbbccefd8ef99131a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:40:56.739330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:56.739971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1353a06-8656-4b15-bce5-94d310b1eed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:40:56.741604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 987/1994] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:40:56 2022] DELETE /identity/v3/projects/df2beeefd80a4cbdbbccefd8ef99131a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:56.746481 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.746900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.747251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.767781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:56.769233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.824760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:56.835520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1008/1995] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:56.843625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.843910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.844213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.865354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:56.866564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:56.880099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 988/1996] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:56.884551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.895489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901787-edec-4179-ae97-09c5e6a9708f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.900185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.900822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.901128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.901401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.902067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:56.902355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.903991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.916763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1009/1997] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:56.921128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.931590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad6355-692e-4f24-b7b4-016539386dfb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:56.937108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:56.937720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:56.938020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:56.938293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:56.939690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:56.941811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:56.974303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 989/1998] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:56 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:56.978506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:56.989622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386d73-832a-4746-bf43-8536f5085a2b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.059540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.060233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.060660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.060999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.061825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.064913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.088592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1010/1999] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:56 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:57.092607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.099899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:57.103514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf94f-5863-4e22-80ea-e6b0a7e86452 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.109438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.110053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.110505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.110975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.111731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.113540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.123520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 990/2000] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:57.128215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.138506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d824ce-7125-4c32-aaac-0748b1f6369d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.144126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.144871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.145356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.145750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.146403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.148232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c859f4-2661-4142-a081-c58f78efa5ea admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.159311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1011/2001] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:57.165024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.175179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e20ada-188b-4ce9-9247-d84c1e2b863d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.180404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.181140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.181493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.181841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] PATH_INFO: `/v3/projects/0ad905c0f6ab4a79bca2d976c05d1c41/users/934391322c1a4c53b7cb3d3a5dae3542/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.201017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad905c0f6ab4a79bca2d976c05d1c41, user_id=934391322c1a4c53b7cb3d3a5dae3542, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.205482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54491e6-7a41-4fab-b75d-97dce2002062 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.231064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 991/2002] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:57 2022] PUT /identity/v3/projects/0ad905c0f6ab4a79bca2d976c05d1c41/users/934391322c1a4c53b7cb3d3a5dae3542/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:57.233188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.233477 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.233815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.243502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.243840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.244088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.279947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] MFA Rules not processed for user `8c1d30d457d74c4e870eba4e2eb9384f`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:57.282240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:57.283166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] MFA Rules not processed for user `934391322c1a4c53b7cb3d3a5dae3542`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:57.284642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:57.386549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 992/2003] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:57.399713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.401319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:57.402197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1012/2004] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:57.406401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.407764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:57.467005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a454389-ba17-4638-8f11-1f87f47b6d24 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.468759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:57.471182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.471802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.472083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.472349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.475680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a92a-1df0-4c60-97a4-c223b3f3b8fd None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.481982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.483107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.483455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.483794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.484080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.484360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.485588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.487717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.493369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 993/2005] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:57.529943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1013/2006] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:40:57 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:40:57.531179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.542295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b896-ac2a-4aba-93a8-b654dbdec820 service glance] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.610936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.611543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.611816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.612090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.612767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.614756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.624493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:57.638501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 994/2007] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:57 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:40:57.642803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.653367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25325d-1d48-489a-9543-bc86eacd5ec7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.658787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.659383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.659664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.659940 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.660673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.662152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.673532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1014/2008] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:40:57.677831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.688693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbc85b-dd96-4ba1-a638-2370834ac04a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.693459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.694120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.694398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.694760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] PATH_INFO: `/v3/projects/1785aca198404e92af922a86497b1275/users/e59dca975da9490db6c2d1004f5ba17a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.710635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1785aca198404e92af922a86497b1275, user_id=e59dca975da9490db6c2d1004f5ba17a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.713340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.736499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 995/2009] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:57 2022] PUT /identity/v3/projects/1785aca198404e92af922a86497b1275/users/e59dca975da9490db6c2d1004f5ba17a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:57.737668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.752538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77415ca7-567c-4b69-a1ea-a5706b6a56a0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.828229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.828872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.829153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.831100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.832470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.833636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adf9b4-5299-49cb-89ee-f4b5090b33de admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.846330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1015/2010] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:57.850729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:57.862145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9018-03a8-41cd-891d-b3381aa23be5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:57.867055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:57.867721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:57.868358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:57.868358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] PATH_INFO: `/v3/projects/1785aca198404e92af922a86497b1275/users/e59dca975da9490db6c2d1004f5ba17a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:57.870231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1785aca198404e92af922a86497b1275, user_id=e59dca975da9490db6c2d1004f5ba17a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:57.873025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:57.888759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 996/2011] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:40:57 2022] PUT /identity/v3/projects/1785aca198404e92af922a86497b1275/users/e59dca975da9490db6c2d1004f5ba17a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:58.569603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:58.570067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:58.570158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:58.595319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:58.596549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:58.675692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1016/2012] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:40:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:58.692156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:58.694560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:58.820343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109954d-eb14-48d4-a354-128f0207af56 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:58.822932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:58.824643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:58.827650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:58.829945 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:58.830440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:58.831120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:58.843493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:58.846530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb66ea-417c-4221-a3ec-390003180dff service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:58.857595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 997/2013] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:40:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:40:58.920707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:58.921051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:58.921555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:58.983179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698e00-433d-4c79-b41a-db4aa6bf8943 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:58.991196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:58.995812 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] MFA Rules not processed for user `e59dca975da9490db6c2d1004f5ba17a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:58.997105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:58.998243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:58.998756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:58.999228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:59.005401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:59.073693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:59.146482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:59.148026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:59.162444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1017/2014] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:40:59.166327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 998/2015] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:40:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:40:59.412035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:59.413659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:59.425423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2e92-87c5-4fd9-a2b3-018f624133e2 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:59.433866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:59.434914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:59.435196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:59.435469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:59.498650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:59.502768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:59.512818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1018/2016] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:40:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:01.105466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:01.115799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c2d06-c174-438f-a335-8a10850c83f5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:01.119540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:01.120255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:01.120576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:01.120912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:01.130849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:01.132372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0238fef-2d5b-48a2-bea2-86e5cd363a61 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:01.142638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 999/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:01.684086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:01.685501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:01.753015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90cca-ad14-4063-9cd1-7ca6c360a395 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:01.757746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:01.758394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:01.758676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:01.758984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] PATH_INFO: `/v3/users/cb0c9d05f53345878f134eea1f61dd50` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:01.760117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0c9d05f53345878f134eea1f61dd50)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:01.762127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:01.816536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0c9d05f53345878f134eea1f61dd50'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:01.817215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0c9d05f53345878f134eea1f61dd50'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:01.817895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0c9d05f53345878f134eea1f61dd50'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:01.831399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0c9d05f53345878f134eea1f61dd50'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:01.833272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:01.841266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0c9d05f53345878f134eea1f61dd50'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:01.864022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1019/2018] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:01 2022] DELETE /identity/v3/users/cb0c9d05f53345878f134eea1f61dd50 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:01.868600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:01.868952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:01.869316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:01.887728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:01.888914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:01.898430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:01.950053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1000/2019] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:01.959992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:01.961446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:02.020754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d82cc7-5f93-4d42-8678-a893a8f535c8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:02.024751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:02.025444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:02.025747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:02.025930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:02.041151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:02.042616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:02.052875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1020/2020] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:02.315436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:02.316980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:02.331481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522f6a-47f1-4506-b777-91ede6b406d9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:02.336592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:02.337270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:02.337550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:02.337994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] PATH_INFO: `/v3/projects/4e32cf82cb92435f900fd6f1baf25faa` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:02.339171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e32cf82cb92435f900fd6f1baf25faa)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:02.341239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:02.389365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e32cf82cb92435f900fd6f1baf25faa', 'request_id': 'req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.396616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e32cf82cb92435f900fd6f1baf25faa', 'request_id': 'req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.403681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e32cf82cb92435f900fd6f1baf25faa', 'request_id': 'req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.415076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] The token cache is being invalidate because project 4e32cf82cb92435f900fd6f1baf25faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:02.415364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.416003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.417494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1001/2021] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:02 2022] DELETE /identity/v3/projects/4e32cf82cb92435f900fd6f1baf25faa => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:02.421300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:02.431989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8328-bf34-4e9d-aa74-205ef8a797a5 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:02.495073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:02.495715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:02.496027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:02.496315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] PATH_INFO: `/v3/users/1cce4bb0eed249969d75b5b2e6e15b3a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:02.497554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cce4bb0eed249969d75b5b2e6e15b3a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:02.499537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:02.548801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cce4bb0eed249969d75b5b2e6e15b3a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.549443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cce4bb0eed249969d75b5b2e6e15b3a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.550047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cce4bb0eed249969d75b5b2e6e15b3a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.562999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cce4bb0eed249969d75b5b2e6e15b3a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.570747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-761457b0-3e60-4e0d-b4aa-91f204cb1cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cce4bb0eed249969d75b5b2e6e15b3a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:02.585218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1021/2022] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:02 2022] DELETE /identity/v3/users/1cce4bb0eed249969d75b5b2e6e15b3a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:02.956718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:02.958244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:02.971134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6a47c-c88c-4eb3-9cac-720562bd7b7d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:03.035055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:03.036476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.036776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.037332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] PATH_INFO: `/v3/users/f5c9b325a5604a6aa427059fdd7f14a2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.038532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c9b325a5604a6aa427059fdd7f14a2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:03.041548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:03.091958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:03.097461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c9b325a5604a6aa427059fdd7f14a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.098133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c9b325a5604a6aa427059fdd7f14a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.106148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c9b325a5604a6aa427059fdd7f14a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.106859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c9b325a5604a6aa427059fdd7f14a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.119629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c9b325a5604a6aa427059fdd7f14a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.134470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1002/2023] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:02 2022] DELETE /identity/v3/users/f5c9b325a5604a6aa427059fdd7f14a2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:03.139171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.139714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.139806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.148616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:03.161630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:03.163601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.164808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:03.166642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.235639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1022/2024] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:03.242187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6fa0-9c1d-4562-9cca-d061e9aac4b2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:03.245216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:03.246662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.247388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:03.248056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.248340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.248614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] PATH_INFO: `/v3/projects/6f4d803a439344d18e41661a9c8988e2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.249780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4d803a439344d18e41661a9c8988e2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:03.252304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:03.302214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4d803a439344d18e41661a9c8988e2', 'request_id': 'req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.306062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d7db9-a407-4dc3-9f8f-8c7934ae39b0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:03.312876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4d803a439344d18e41661a9c8988e2', 'request_id': 'req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.320207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4d803a439344d18e41661a9c8988e2', 'request_id': 'req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.331921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] The token cache is being invalidate because project 6f4d803a439344d18e41661a9c8988e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:03.332404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.333081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.334708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1003/2025] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:03 2022] DELETE /identity/v3/projects/6f4d803a439344d18e41661a9c8988e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:03.338346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:03.339699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.357783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:03.358413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.358729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.358997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.361360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.390631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:03.408195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5a05a-0c05-4c19-b77f-581efaafcf7d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:03.414005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:03.414804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.415091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.415609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] PATH_INFO: `/v3/users/abd26a3371eb46e0acdc19fa2ad69af0` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.416744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd26a3371eb46e0acdc19fa2ad69af0)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:03.418648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:03.429197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:03.430629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:03.441358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1023/2026] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:03.467137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd26a3371eb46e0acdc19fa2ad69af0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.467781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd26a3371eb46e0acdc19fa2ad69af0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.474959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd26a3371eb46e0acdc19fa2ad69af0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.475683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd26a3371eb46e0acdc19fa2ad69af0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.488961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd26a3371eb46e0acdc19fa2ad69af0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:03.511262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1004/2027] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:03 2022] DELETE /identity/v3/users/abd26a3371eb46e0acdc19fa2ad69af0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:03.819315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:03.822179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.840992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584160c-90d2-415a-96cf-9f32b10d2689 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:03.870946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:03.920532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:03.923547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:03.943774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:03.944066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:03.944349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:03.944675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] PATH_INFO: `/v3/projects/c9deb08d60024fdebf1af8998a2ecf19` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:03.946070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9deb08d60024fdebf1af8998a2ecf19)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:03.948266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.047351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d341a1-1b5e-4e5c-a68b-0b6d128aa8c0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.074661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9deb08d60024fdebf1af8998a2ecf19', 'request_id': 'req-398a0e39-7f8c-4259-8706-a7a8aeaf7354'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.084289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9deb08d60024fdebf1af8998a2ecf19', 'request_id': 'req-398a0e39-7f8c-4259-8706-a7a8aeaf7354'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.091902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9deb08d60024fdebf1af8998a2ecf19', 'request_id': 'req-398a0e39-7f8c-4259-8706-a7a8aeaf7354'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.110206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] The token cache is being invalidate because project c9deb08d60024fdebf1af8998a2ecf19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:04.110645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.111397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.113404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1024/2028] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:03 2022] DELETE /identity/v3/projects/c9deb08d60024fdebf1af8998a2ecf19 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:04.117259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.120801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.137005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.138263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.138883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.139168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] PATH_INFO: `/v3/projects/ee727636cce944b5894a8aafe96dc08d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.140431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee727636cce944b5894a8aafe96dc08d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.143276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.172587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:04.204159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0e39-7f8c-4259-8706-a7a8aeaf7354 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.208902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee727636cce944b5894a8aafe96dc08d', 'request_id': 'req-b29e8753-9fff-4406-9794-200111261e84'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.218733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee727636cce944b5894a8aafe96dc08d', 'request_id': 'req-b29e8753-9fff-4406-9794-200111261e84'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.228355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee727636cce944b5894a8aafe96dc08d', 'request_id': 'req-b29e8753-9fff-4406-9794-200111261e84'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.247510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] The token cache is being invalidate because project ee727636cce944b5894a8aafe96dc08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:04.247803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.248452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b29e8753-9fff-4406-9794-200111261e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.249935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1005/2029] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:03 2022] DELETE /identity/v3/projects/ee727636cce944b5894a8aafe96dc08d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:04.253526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:04.256355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85cc8-45dc-4b06-a9c1-2708412ee2b8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.256667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85cc8-45dc-4b06-a9c1-2708412ee2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.256981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85cc8-45dc-4b06-a9c1-2708412ee2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.268730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.269369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.269654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.269932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] PATH_INFO: `/v3/users/7ee8e4cea79e430e907d74b4df3feec9` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.271124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee8e4cea79e430e907d74b4df3feec9)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.273123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.277485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0bb85cc8-45dc-4b06-a9c1-2708412ee2b8 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:04.278710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0bb85cc8-45dc-4b06-a9c1-2708412ee2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.287482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1006/2030] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:04.295266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.295550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.295842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.317546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:04.318787 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.322082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee8e4cea79e430e907d74b4df3feec9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.322777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee8e4cea79e430e907d74b4df3feec9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.323416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee8e4cea79e430e907d74b4df3feec9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.326719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1007/2031] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:04.331359 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.336674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee8e4cea79e430e907d74b4df3feec9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.340471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915985c2-d603-4b4f-9345-75bad2a1583b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.346533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.346862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee8e4cea79e430e907d74b4df3feec9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.347185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.347590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.347908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.348640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:04.348948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.350545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.360664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1025/2032] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:04 2022] DELETE /identity/v3/users/7ee8e4cea79e430e907d74b4df3feec9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:04.366010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1008/2033] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:41:04.370330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.443937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08037b0-a904-4307-874f-8f1d0e42a21d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.448792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.449424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.449700 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.449971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.450589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.452532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.487943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1026/2034] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:04.489586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.500897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649abc1-dc35-4d3b-958d-26767b82683c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.568205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.568821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.569105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.569379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.570441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.572280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.596212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1009/2035] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:04.600647 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.611261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf24b19-dec9-471f-98ec-3c04462ef306 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.616510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.617275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.619871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.619871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.619871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.620243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.635780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1027/2036] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:04.640313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.647765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.655036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.655444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759795-9b23-4906-8a13-b851a9d888ab admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.663117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.663852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.664140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.664413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.665098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.667159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.671211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664b0a4-19aa-45c3-a1a1-ebd006b5a989 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.676429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.676984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.677266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.677543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] PATH_INFO: `/v3/projects/ad192f9f8903442f9e1b3284029916e8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.679088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad192f9f8903442f9e1b3284029916e8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.680997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.681122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1010/2037] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:04.684936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.695355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec27a61-b480-461b-b0f5-95405f1a3642 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.700184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.700833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.701112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.701388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] PATH_INFO: `/v3/projects/2d9b2db22b4342f3b6b8f2caabdb9ace/users/644011d074b848c7bf2c8435e60dc6ca/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.718419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9b2db22b4342f3b6b8f2caabdb9ace, user_id=644011d074b848c7bf2c8435e60dc6ca, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.721126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fa4bb-753c-46e4-bc78-960708f6b5c6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.725037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:04.736460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad192f9f8903442f9e1b3284029916e8', 'request_id': 'req-22b58097-bb90-493c-b078-e898cf312775'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.746802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1011/2038] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:04 2022] PUT /identity/v3/projects/2d9b2db22b4342f3b6b8f2caabdb9ace/users/644011d074b848c7bf2c8435e60dc6ca/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:04.753073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad192f9f8903442f9e1b3284029916e8', 'request_id': 'req-22b58097-bb90-493c-b078-e898cf312775'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.753360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.754101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.754473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.761592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad192f9f8903442f9e1b3284029916e8', 'request_id': 'req-22b58097-bb90-493c-b078-e898cf312775'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.773561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] The token cache is being invalidate because project ad192f9f8903442f9e1b3284029916e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:04.773987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.774782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-22b58097-bb90-493c-b078-e898cf312775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:04.776621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1028/2039] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:04 2022] DELETE /identity/v3/projects/ad192f9f8903442f9e1b3284029916e8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:04.781372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351adb-6f00-4dba-8ccf-e8ca015b6d4d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.781755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351adb-6f00-4dba-8ccf-e8ca015b6d4d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.782150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351adb-6f00-4dba-8ccf-e8ca015b6d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.802794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] MFA Rules not processed for user `644011d074b848c7bf2c8435e60dc6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:04.804427 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.809025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-bb351adb-6f00-4dba-8ccf-e8ca015b6d4d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:04.810492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-bb351adb-6f00-4dba-8ccf-e8ca015b6d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.828206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:04.873342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1029/2040] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:04.881767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.882084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.882435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.889468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1012/2041] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:04.893868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.895214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.903260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:04.904444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:04.908376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76e17-aefe-490f-a718-596ecf4f4cc6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.912490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1030/2042] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:04.912927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.913545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.913848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.914102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.914756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.916632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.916737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.926279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367fefd-0a63-4614-8b73-5f6835b584f3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:04.932635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:04.933233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:04.933513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:04.933816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:04.934463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:04.934853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:04.936870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:04.945340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1013/2043] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:04.949120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.952961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1031/2044] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:41:04.953950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:04.964915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1049ce-40c5-4a08-80f3-b6969a582e68 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.013482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42fe8-992d-4535-9b4d-368968ff4fe0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.018572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.019208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.019510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.019789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.020410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.021433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.022040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.022320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.022415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.022592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.023250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.025143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.048972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1014/2045] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:05.050641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.058023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1032/2046] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:04 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:05.064483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.078797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa879e-9a2b-45d4-9cf0-ab54cb1b51f6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.100570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:05.137334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92fd5-d393-432f-a578-07907cd0647e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.142601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.143536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.143904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.144278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.144530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.145132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.145238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.145412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.145684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.146321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.147617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.147924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.158154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1015/2047] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:05.162097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.169597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1033/2048] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:05.171567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a542b8-735d-4790-bd53-f1614eebd6f7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.174545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.177459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.178021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.178559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.178865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.179513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.181971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.184875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a8693-c903-4bc1-b5e2-dba761fb3cca admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.190429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.190960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.191244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.191527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.192305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.193913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.194250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1016/2049] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:05.198167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.208632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1034/2050] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:05.209651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.211264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696c5a-482a-47e2-a367-e4b78887445a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.217980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.218645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.218954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.219772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] PATH_INFO: `/v3/projects/1c7a7a89d33a4c99bf5b35f213dcb1d6/users/753f273ea62b42d7b92568f39cb3fb49/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.219881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20f28e-55d6-45aa-8b84-b2b850122612 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.225753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.226357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.226652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.226952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.227602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.229672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.232899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:05.239242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7a7a89d33a4c99bf5b35f213dcb1d6, user_id=753f273ea62b42d7b92568f39cb3fb49, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.242207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e584b-eb78-4921-806f-ef3cd8704555 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.243184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1035/2051] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:05.248112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.260381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e642-c3ea-400a-a6d4-764fd500ba3f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.261522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1017/2052] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:05 2022] PUT /identity/v3/projects/1c7a7a89d33a4c99bf5b35f213dcb1d6/users/753f273ea62b42d7b92568f39cb3fb49/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:05.276587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.276854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.277129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.325872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] MFA Rules not processed for user `753f273ea62b42d7b92568f39cb3fb49`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:05.327290 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:05.384564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.385223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.385496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.385763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] PATH_INFO: `/v3/projects/bc24b118be6e43c4965862662cead294/users/bd63094a8b8d480b9a6504380937e890/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.399782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1018/2053] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:05.403547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc24b118be6e43c4965862662cead294, user_id=bd63094a8b8d480b9a6504380937e890, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.404389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.406360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba218ecc-5c5f-4d15-8148-d83fe43bd5d5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.417815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eba8b1-3807-452c-9da4-1766bcdcd858 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.426266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1036/2054] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:05 2022] PUT /identity/v3/projects/bc24b118be6e43c4965862662cead294/users/bd63094a8b8d480b9a6504380937e890/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:05.437444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.437742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.438053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.474248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] MFA Rules not processed for user `bd63094a8b8d480b9a6504380937e890`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:05.475593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:05.483664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.484382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.484748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.485674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.486334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.488402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.521393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1019/2055] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:05.525953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.554177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1037/2056] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:05.559476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.571623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c8680-d42a-4ce8-aa58-4da125682440 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.592018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:05.624055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d38dd4-dab6-4e1e-b4e0-1b7c54e53489 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.646974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.646974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.646974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.646974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.647284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.648181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.673893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.674658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.675312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.675853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.676407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.676887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:05.680201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.692025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1020/2057] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:05.696717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.709802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dfb43-f63d-4720-bfbe-227fcf1a96f7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.712379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1038/2058] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:05.714591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.715371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.715657 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.715957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.716365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.716572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.718206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.730367 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1021/2059] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:05.734539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.745156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534ee9-8e75-4c2b-9073-92b99124b4bf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.780901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76209c20-a4aa-481e-90a0-d89b390eb218 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.786332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.786977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.787268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.787545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.788176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.790116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.810176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.810897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.811171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.811448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] PATH_INFO: `/v3/projects/a68d33bd54244c91a2b4d59319c14112/users/4c9329118efd43fa87b8fc408786e3c5/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.819405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1039/2060] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:05 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:05.821427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.832454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9542-9dfa-482f-86e3-b7ff8701e002 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.837158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.837771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68d33bd54244c91a2b4d59319c14112, user_id=4c9329118efd43fa87b8fc408786e3c5, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.837885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.838076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.838348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.839010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.840577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.840679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.854751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1040/2061] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:05.858023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1022/2062] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:05 2022] PUT /identity/v3/projects/a68d33bd54244c91a2b4d59319c14112/users/4c9329118efd43fa87b8fc408786e3c5/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:05.859889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.861898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.873979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57d57d-e914-4dfc-9ac2-54fc9342a2a2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.944071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.944307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a55c2-029b-45d0-b23a-e398d3d97104 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.944629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.944903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.945176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] PATH_INFO: `/v3/projects/1bbad74341bd417ba020594abc2bf732/users/0a1e808e9783463e8cf634992cae0e3d/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.948842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:05.949445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:05.949724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:05.950004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:05.950647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.952275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.961862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbad74341bd417ba020594abc2bf732, user_id=0a1e808e9783463e8cf634992cae0e3d, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:05.963024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1023/2063] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:05.964535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:05.967131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:05.973260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:05.978519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dff37d-a5d4-484a-adf5-05936006a297 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:05.979845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1041/2064] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:05 2022] PUT /identity/v3/projects/1bbad74341bd417ba020594abc2bf732/users/0a1e808e9783463e8cf634992cae0e3d/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:05.983692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.044509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.045185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.045467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.045736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] PATH_INFO: `/v3/projects/a68d33bd54244c91a2b4d59319c14112/users/4c9329118efd43fa87b8fc408786e3c5/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.047722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68d33bd54244c91a2b4d59319c14112, user_id=4c9329118efd43fa87b8fc408786e3c5, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.050391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a582e7-aba6-4c9a-9b74-566c47d07935 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.050577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16b2ef-3078-41a2-b92f-a4a045426d5f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.055658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.056357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.056625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:06.056867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.057176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.058133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.060332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.062593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1024/2065] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:05 2022] PUT /identity/v3/projects/a68d33bd54244c91a2b4d59319c14112/users/4c9329118efd43fa87b8fc408786e3c5/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:06.074068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.074356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.074662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.077064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1042/2066] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:06.077900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.091585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8063b-c08e-4b55-bbfe-2f2b48072db0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.104000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] MFA Rules not processed for user `4c9329118efd43fa87b8fc408786e3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:06.105240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:06.157698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.158364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.158650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.159247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] PATH_INFO: `/v3/projects/1bbad74341bd417ba020594abc2bf732/users/0a1e808e9783463e8cf634992cae0e3d/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.160924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbad74341bd417ba020594abc2bf732, user_id=0a1e808e9783463e8cf634992cae0e3d, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.163929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd300eb-c5b6-4ca0-8885-bbaa7a7dd6b5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.176837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1043/2067] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:06 2022] PUT /identity/v3/projects/1bbad74341bd417ba020594abc2bf732/users/0a1e808e9783463e8cf634992cae0e3d/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:06.186422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1025/2068] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:06.195539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.195851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.196191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.197707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.199163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:06.224565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] MFA Rules not processed for user `0a1e808e9783463e8cf634992cae0e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:06.225763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:06.257806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a88907-93f7-419e-ab21-eb9f4c25a2f4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.298715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1044/2069] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:06.311795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.312840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.313240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:06.313487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.313771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.314217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.324948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ec9e1-0b47-4d95-bf8f-249f4c7eb54d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.329092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.329679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.329953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.330335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.339795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.341043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.347168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:06.353016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1045/2070] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:06.366987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:06.382109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.383614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.404020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1026/2071] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:06.449924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.453909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:06.462273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468f89f-70e0-41ad-ad88-d1079a8cbaf2 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.462907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca1371-fe0d-4825-9d4a-b0e830854354 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:06.467334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.468160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.468457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.468948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:06.470059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.470197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:06.470548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:06.471011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:06.549220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.550834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.560502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1027/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:06.574129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:06.575818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fb689-1660-4ea2-b1b8-7bb7cab201b3 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:06.589844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1046/2073] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:07.174611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:07.238183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734a79-bd71-496d-ab2c-d1d765ccaf3b service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:07.242674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:07.243435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:07.243713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:07.243985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] PATH_INFO: `/v3/users/bd63094a8b8d480b9a6504380937e890` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:07.245105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd63094a8b8d480b9a6504380937e890)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:07.247070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:07.297065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd63094a8b8d480b9a6504380937e890'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.298263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd63094a8b8d480b9a6504380937e890'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.311531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd63094a8b8d480b9a6504380937e890'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.313465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd63094a8b8d480b9a6504380937e890'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.326109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd63094a8b8d480b9a6504380937e890'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.342713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1028/2074] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:07 2022] DELETE /identity/v3/users/bd63094a8b8d480b9a6504380937e890 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:07.347506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:07.347963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:07.348172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:07.369819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:07.371486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:07.437041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1047/2075] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:07.449699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:07.451461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:07.464127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:07.516870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3251-3198-44dc-bd84-7d4abbfe575c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:07.520695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:07.521355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:07.521630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:07.521897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:07.531414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:07.533377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:07.543600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1029/2076] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:07.864458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:07.866055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:07.886463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4e965-f67d-4cf7-a7d4-d25aa53557b4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:07.891772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:07.892405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:07.892682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:07.892955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] PATH_INFO: `/v3/projects/bc24b118be6e43c4965862662cead294` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:07.894139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc24b118be6e43c4965862662cead294)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:07.896075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:07.943252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc24b118be6e43c4965862662cead294', 'request_id': 'req-9c1bf3f5-4e97-43df-be88-dde13cc22376'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.951226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc24b118be6e43c4965862662cead294', 'request_id': 'req-9c1bf3f5-4e97-43df-be88-dde13cc22376'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.958554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc24b118be6e43c4965862662cead294', 'request_id': 'req-9c1bf3f5-4e97-43df-be88-dde13cc22376'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.974827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] The token cache is being invalidate because project bc24b118be6e43c4965862662cead294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:07.975117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.975764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:07.977244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1048/2077] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:07 2022] DELETE /identity/v3/projects/bc24b118be6e43c4965862662cead294 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:07.981260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:07.993041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31f8e-75da-4a4a-af9c-142f51203abc service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:08.055802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:08.056438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.056713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.056986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] PATH_INFO: `/v3/users/0a1e808e9783463e8cf634992cae0e3d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.058260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1e808e9783463e8cf634992cae0e3d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:08.060395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:08.112204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1e808e9783463e8cf634992cae0e3d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.112999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1e808e9783463e8cf634992cae0e3d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.121557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1e808e9783463e8cf634992cae0e3d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.122627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1e808e9783463e8cf634992cae0e3d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.136970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-85741581-23c8-4a69-a0ad-c94e514eae07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1e808e9783463e8cf634992cae0e3d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.151849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1030/2078] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:07 2022] DELETE /identity/v3/users/0a1e808e9783463e8cf634992cae0e3d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:08.505611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:08.507456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:08.525871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf3f5-4e97-43df-be88-dde13cc22376 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:08.551796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:08.607033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:08.607680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.607960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.608231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] PATH_INFO: `/v3/projects/1bbad74341bd417ba020594abc2bf732` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.609343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbad74341bd417ba020594abc2bf732)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:08.612072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:08.656936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbad74341bd417ba020594abc2bf732', 'request_id': 'req-8b7c4cde-c4d9-466d-b703-170fd6b00260'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.664332 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbad74341bd417ba020594abc2bf732', 'request_id': 'req-8b7c4cde-c4d9-466d-b703-170fd6b00260'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.671148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbad74341bd417ba020594abc2bf732', 'request_id': 'req-8b7c4cde-c4d9-466d-b703-170fd6b00260'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.684798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] The token cache is being invalidate because project 1bbad74341bd417ba020594abc2bf732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:08.684798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.685044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8b7c4cde-c4d9-466d-b703-170fd6b00260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:08.686511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1049/2079] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:08 2022] DELETE /identity/v3/projects/1bbad74341bd417ba020594abc2bf732 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:08.692169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.692608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.692783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.711895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:08.713090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:08.738000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:08.779530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1031/2080] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:08.787743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.787743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.788690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.809699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:08.810929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:08.819768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1050/2081] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:08.824226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:08.835423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb31f82-b695-4dab-8cb8-e0e5d437ade7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:08.841616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:08.842501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.842848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.843125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.843775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:08.844110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:08.845702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:08.858279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1032/2082] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:08.865108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:08.874468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef02ead-246f-47bb-80a9-9d5df92d266a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:08.880252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:08.880955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.881229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.881496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:08.882167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:08.884169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:08.915514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1051/2083] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:08 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:08.919320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:08.931027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1b3b7-aeed-4eb8-b23f-84460bc52ffb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:08.998686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:08.999350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:08.999631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:08.999903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.000616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.002580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.024337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1033/2084] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:08 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:09.028577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.040297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5c2c-e3ef-4ce0-be30-a8fe8cb2775a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.045211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.045866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.046142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.046458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.047188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.049402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.060597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1052/2085] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:09.065759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.077564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965e5a-02e8-4a81-8d6c-a06335489cdf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.082461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.083725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.084019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.084302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.085137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.086847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27684e8-0386-4430-b704-33d8154c8405 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.099414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1034/2086] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:09.104226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.116254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b74a9-e437-4b91-af4d-b7900942c4e9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.120815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.121464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.121740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.122035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] PATH_INFO: `/v3/projects/db234dbd92074ebd96d224bda604e97f/users/ef2a8d2134c941d4b0b39c576f717be3/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.137358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db234dbd92074ebd96d224bda604e97f, user_id=ef2a8d2134c941d4b0b39c576f717be3, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.140164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.152911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1053/2087] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:09 2022] PUT /identity/v3/projects/db234dbd92074ebd96d224bda604e97f/users/ef2a8d2134c941d4b0b39c576f717be3/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:09.166595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.166904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.167177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.194937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] MFA Rules not processed for user `ef2a8d2134c941d4b0b39c576f717be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:09.196122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:09.264543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1035/2088] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:09.276324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.278030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:09.342991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:09.347667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be0b61-0845-40c7-95aa-63d098cf2455 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.352954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.353807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.354241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.354565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.363188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.364613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153c450-6a78-4e3a-b758-a2d59b56d5fb service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.373246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1054/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:09.394840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.423196 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:09.460705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e67242-4d4d-414c-a6a3-3a6d487af5f8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.465336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.465960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.466238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.466513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] PATH_INFO: `/v3/users/ef2a8d2134c941d4b0b39c576f717be3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.467665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2a8d2134c941d4b0b39c576f717be3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.470132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.517111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2a8d2134c941d4b0b39c576f717be3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.517764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2a8d2134c941d4b0b39c576f717be3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.525235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2a8d2134c941d4b0b39c576f717be3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.526113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2a8d2134c941d4b0b39c576f717be3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.539105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2a8d2134c941d4b0b39c576f717be3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.552676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1036/2090] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:09 2022] DELETE /identity/v3/users/ef2a8d2134c941d4b0b39c576f717be3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:09.557518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.558021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.558131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.577683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:09.579475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:09.662504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1055/2091] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:09.672711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.675626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:09.743589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:09.745157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:09.764417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7a240-f029-46db-ab92-17a222d14cf4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.764828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13240420-5293-4655-9dc3-f9065172e8fe admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:09.770668 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.770958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:09.771675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.771889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.772176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] PATH_INFO: `/v3/users/644011d074b848c7bf2c8435e60dc6ca` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.772434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.773434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644011d074b848c7bf2c8435e60dc6ca)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.773869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.774295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.776680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.790475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:09.792825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7695c-2102-461c-a278-e6a2c7925ae7 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:09.808315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1037/2092] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:09.865331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644011d074b848c7bf2c8435e60dc6ca'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.866285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644011d074b848c7bf2c8435e60dc6ca'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.868410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644011d074b848c7bf2c8435e60dc6ca'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.889664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644011d074b848c7bf2c8435e60dc6ca'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.901976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644011d074b848c7bf2c8435e60dc6ca'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:09.924722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1056/2093] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:09 2022] DELETE /identity/v3/users/644011d074b848c7bf2c8435e60dc6ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:09.930170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:09.930725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:09.931159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:09.957469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:09.958875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.025931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:10.069278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1038/2094] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:10.114936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.116486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.215071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec8a93-abb7-484f-be1f-67176f0b5886 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.218580 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.220140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.220810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.221285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.235247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.237443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e722-2e0d-4823-ad77-2d2f88be7818 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.256425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1057/2095] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:10.283984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.286010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.299722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d316db6-c51c-4cce-b0a0-10606fc2ee1d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.306360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.307360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.307761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.308632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] PATH_INFO: `/v3/projects/db234dbd92074ebd96d224bda604e97f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.328088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] RBAC: Authorizing `identity:delete_project(project_id=db234dbd92074ebd96d224bda604e97f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.330816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.402386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db234dbd92074ebd96d224bda604e97f', 'request_id': 'req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.420586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db234dbd92074ebd96d224bda604e97f', 'request_id': 'req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.432224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db234dbd92074ebd96d224bda604e97f', 'request_id': 'req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.454293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] The token cache is being invalidate because project db234dbd92074ebd96d224bda604e97f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:10.454995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.455868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-535a4cdb-fe4a-49f0-aeb8-97808ef1c1df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.457630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1039/2096] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:10 2022] DELETE /identity/v3/projects/db234dbd92074ebd96d224bda604e97f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:10.463524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.463983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.465415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.504686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:10.516933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:10.519030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.610818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1058/2097] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:10.618852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.619140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.619417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.659844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:10.660424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.661250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.662905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.673778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1040/2098] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:10.676436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.680961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61214ee-725d-4985-b794-f97d10dfe073 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.685621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.686369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.686534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.686857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] PATH_INFO: `/v3/projects/2d9b2db22b4342f3b6b8f2caabdb9ace` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.688158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9b2db22b4342f3b6b8f2caabdb9ace)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.689850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416496a-136a-47a3-9517-cb1bada27915 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.690110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.695737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.696679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.697374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.697652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.698309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:10.698610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.700689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.714791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1041/2099] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:10.716515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.727559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a94e-781b-4128-a28a-3d938bd52387 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.740595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9b2db22b4342f3b6b8f2caabdb9ace', 'request_id': 'req-741800c3-d8ef-4b17-bfd9-79f392c9e669'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.748415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9b2db22b4342f3b6b8f2caabdb9ace', 'request_id': 'req-741800c3-d8ef-4b17-bfd9-79f392c9e669'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.756274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9b2db22b4342f3b6b8f2caabdb9ace', 'request_id': 'req-741800c3-d8ef-4b17-bfd9-79f392c9e669'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.780341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] The token cache is being invalidate because project 2d9b2db22b4342f3b6b8f2caabdb9ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:10.780893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.781628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.787094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1059/2100] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:10 2022] DELETE /identity/v3/projects/2d9b2db22b4342f3b6b8f2caabdb9ace => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:10.788287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.789694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.792424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.793036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.793317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.793588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.794217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.796243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.804815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741800c3-d8ef-4b17-bfd9-79f392c9e669 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:10.809731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:10.810363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:10.810641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:10.810964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] PATH_INFO: `/v3/users/753f273ea62b42d7b92568f39cb3fb49` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:10.812108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753f273ea62b42d7b92568f39cb3fb49)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:10.814235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:10.840131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1042/2101] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:10 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:10.846557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:10.849866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:10.882201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753f273ea62b42d7b92568f39cb3fb49'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.883150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753f273ea62b42d7b92568f39cb3fb49'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.883808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753f273ea62b42d7b92568f39cb3fb49'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.900361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753f273ea62b42d7b92568f39cb3fb49'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.906487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:10.911446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753f273ea62b42d7b92568f39cb3fb49'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:10.930467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1060/2102] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:10 2022] DELETE /identity/v3/users/753f273ea62b42d7b92568f39cb3fb49 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:10.939331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a4fe1-1702-4b50-bdcc-0f51f590e5ad admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.015754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.016918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.017468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.017670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.018308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.020594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.060691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1043/2103] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:10 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:11.066096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.068134 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:11.079944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:11.085851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7cd4c-8ff0-4113-b87a-ac6fae2c53c7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.094189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.095805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.096484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.096936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.098184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.100948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.113414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1061/2104] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:11.117772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.135752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160103bc-aa07-4760-82b7-7188f6d7e9cb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.148793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.155520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.156667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.157640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.158845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.161186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76645ab1-c7c1-4114-814d-1d4ce4c186d8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.178770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1044/2105] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:11.180611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.210611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5da06-c816-478c-880c-9f570b25615d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.238720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.239491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.239757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.240325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] PATH_INFO: `/v3/projects/9fe88de137a54426a53c679c6d093bb1/users/6f649a4547d743a788da2020f8cc768b/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.260597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe88de137a54426a53c679c6d093bb1, user_id=6f649a4547d743a788da2020f8cc768b, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.266728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.291025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1062/2106] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:11 2022] PUT /identity/v3/projects/9fe88de137a54426a53c679c6d093bb1/users/6f649a4547d743a788da2020f8cc768b/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:11.301783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.302313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.302783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.336525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] MFA Rules not processed for user `6f649a4547d743a788da2020f8cc768b`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:11.337760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:11.406424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.415584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:11.434779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1045/2107] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:11.446275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.447861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:11.533281 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b3ea-036c-4b12-8525-2caa6037bdff None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.538927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569752-d7a2-444b-a0f0-d72b083b4e76 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.540805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.542170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.542453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.542766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.551475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:11.556707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.559077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.559392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.559735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.560017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.560294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] PATH_INFO: `/v3/projects/1c7a7a89d33a4c99bf5b35f213dcb1d6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.561453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7a7a89d33a4c99bf5b35f213dcb1d6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.564142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.576910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1046/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:11.631546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7a7a89d33a4c99bf5b35f213dcb1d6', 'request_id': 'req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.640430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7a7a89d33a4c99bf5b35f213dcb1d6', 'request_id': 'req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.648191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7a7a89d33a4c99bf5b35f213dcb1d6', 'request_id': 'req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.669211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] The token cache is being invalidate because project 1c7a7a89d33a4c99bf5b35f213dcb1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:11.669503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.670188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.671749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1063/2109] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:11 2022] DELETE /identity/v3/projects/1c7a7a89d33a4c99bf5b35f213dcb1d6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:11.676419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:11.689535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631d00-6052-476c-b2bd-fd891743bdae service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:11.752538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:11.753361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:11.753713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:11.753989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] PATH_INFO: `/v3/users/4c9329118efd43fa87b8fc408786e3c5` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:11.755160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9329118efd43fa87b8fc408786e3c5)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:11.757079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:11.816864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9329118efd43fa87b8fc408786e3c5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.817548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9329118efd43fa87b8fc408786e3c5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.825420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9329118efd43fa87b8fc408786e3c5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.826221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9329118efd43fa87b8fc408786e3c5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.840084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6326dbf-dede-4f15-b253-73191bfdd770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9329118efd43fa87b8fc408786e3c5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:11.854855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1047/2110] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:11 2022] DELETE /identity/v3/users/4c9329118efd43fa87b8fc408786e3c5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:12.149920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.151470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:12.175889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce710a4-1d71-43d5-8eaf-7d78b4b3276d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.232675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:12.240335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.241302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.241741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.242312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] PATH_INFO: `/v3/projects/a68d33bd54244c91a2b4d59319c14112` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.243957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68d33bd54244c91a2b4d59319c14112)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.246200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.295145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68d33bd54244c91a2b4d59319c14112', 'request_id': 'req-133681a0-d5e0-4554-aae6-755e7432f169'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:12.304517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68d33bd54244c91a2b4d59319c14112', 'request_id': 'req-133681a0-d5e0-4554-aae6-755e7432f169'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:12.311962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68d33bd54244c91a2b4d59319c14112', 'request_id': 'req-133681a0-d5e0-4554-aae6-755e7432f169'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:12.322935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] The token cache is being invalidate because project a68d33bd54244c91a2b4d59319c14112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:12.323226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:12.323879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-133681a0-d5e0-4554-aae6-755e7432f169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:12.325444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1064/2111] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:12 2022] DELETE /identity/v3/projects/a68d33bd54244c91a2b4d59319c14112 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:12.331731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.332024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.332299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.353479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:12.353635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:12.418732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1048/2112] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:12.428031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.428316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.428855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.467188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:12.468279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:12.480680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1065/2113] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:12.486319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.500639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985883a-51b9-42a0-9caf-c06ba89f8153 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.507007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.507625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.507887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.508159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.508802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:12.509106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.510721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.521836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:12.534915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.542815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1049/2114] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:12.548124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fb0e7-530a-433f-95ff-5ed3fda18c89 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.565324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.566800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.567327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.567604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.568391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.570585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.608952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1066/2115] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:12.613157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.626906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de2a8-623f-4987-aef0-64a8624eaf26 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.725537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.726406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.727420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.728036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.729009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.731166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.757929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1050/2116] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:12 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:12.761982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.775219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205a15-6233-444e-a9b8-208d8b5ee17b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.782419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.783516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.784168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.785027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.786257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.788501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.803320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1067/2117] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:12.804885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.818154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1e027-f8db-4dd2-a928-9f011c46c088 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.827031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.827667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.828046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.828404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.830014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.831929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b858a-3f61-4e44-b2b3-f6628313b015 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.845415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1051/2118] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:12.849796 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:12.860849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f39e-41d3-4fdf-b3f8-90fec13735a1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:12.866374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:12.867087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.867374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.867657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] PATH_INFO: `/v3/projects/acde0197f28e4e008c25e840049a5793/users/85ff60ea7a4e41e6b10d0bdfc65ae3ca/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.886800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acde0197f28e4e008c25e840049a5793, user_id=85ff60ea7a4e41e6b10d0bdfc65ae3ca, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:12.889931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:12.910025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1068/2119] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:12 2022] PUT /identity/v3/projects/acde0197f28e4e008c25e840049a5793/users/85ff60ea7a4e41e6b10d0bdfc65ae3ca/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:12.920916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:12.921207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:12.921489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:12.954851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] MFA Rules not processed for user `85ff60ea7a4e41e6b10d0bdfc65ae3ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:12.956148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:13.036869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1052/2120] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:13.048787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:13.050548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:13.129514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95739256-88eb-4565-8ce5-d0d6ad967dd7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:13.133262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:13.133979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:13.134273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:13.134528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:13.143765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:13.145294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73e1ea-dec5-429b-b24a-f64b1b34169e service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:13.154813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1069/2121] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:13.176573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:13.262082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e0b95-4e61-492c-ad10-be378df4dabb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:13.267575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:13.268271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:13.268650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:13.268971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] PATH_INFO: `/v3/users/85ff60ea7a4e41e6b10d0bdfc65ae3ca` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:13.271027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ff60ea7a4e41e6b10d0bdfc65ae3ca)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:13.273732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:13.295217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:13.333753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ff60ea7a4e41e6b10d0bdfc65ae3ca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:13.334418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ff60ea7a4e41e6b10d0bdfc65ae3ca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:13.343243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ff60ea7a4e41e6b10d0bdfc65ae3ca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:13.343949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ff60ea7a4e41e6b10d0bdfc65ae3ca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:13.356564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ff60ea7a4e41e6b10d0bdfc65ae3ca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:13.369294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1053/2122] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:13 2022] DELETE /identity/v3/users/85ff60ea7a4e41e6b10d0bdfc65ae3ca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:13.373794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:13.374077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:13.374386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:13.393706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:13.395043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:13.415742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:13.457007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1070/2123] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:13.467042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:13.468521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:13.537375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852fab2-12ea-4171-8701-7fda4b563006 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:13.540977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:13.541579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:13.541851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:13.542116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:13.550920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:13.552333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b235e-cafc-4ee1-a7fe-5ba57237b575 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:13.562707 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1054/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:14.129863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.131363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:14.147494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea38b8-e4de-4226-bce9-9104786f4a29 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.165900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.166815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.167337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.169877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] PATH_INFO: `/v3/projects/acde0197f28e4e008c25e840049a5793` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.171280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acde0197f28e4e008c25e840049a5793)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.173459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.233171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acde0197f28e4e008c25e840049a5793', 'request_id': 'req-8bafd7d0-850b-42de-87b8-28734d502f91'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:14.251759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acde0197f28e4e008c25e840049a5793', 'request_id': 'req-8bafd7d0-850b-42de-87b8-28734d502f91'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:14.261899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acde0197f28e4e008c25e840049a5793', 'request_id': 'req-8bafd7d0-850b-42de-87b8-28734d502f91'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:14.275462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] The token cache is being invalidate because project acde0197f28e4e008c25e840049a5793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:14.275759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:14.276781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8bafd7d0-850b-42de-87b8-28734d502f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:14.278472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1071/2125] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:14 2022] DELETE /identity/v3/projects/acde0197f28e4e008c25e840049a5793 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:14.284879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.285368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.286743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.316017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:14.317553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:14.399751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1055/2126] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:14.407511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.407913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.408261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.433541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:14.434831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:14.450835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1072/2127] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:14.455320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.466070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2865e6-d34a-40ff-b058-10e8f8b5d908 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.471557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.472421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.472913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.473286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.474039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:14.474434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.476448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.489966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1056/2128] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:14.494013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.507749 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83b69f-342c-41de-ac03-59cf12d4b69b None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.516632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.517759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.518046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.518499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.519624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.521897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.575960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1073/2129] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:14.577425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.596390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1334ae3-61bc-452d-9ec2-71a92d50e14f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.614512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:14.667421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.668041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.668320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.668592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.669219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.671468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.703928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1057/2130] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:14.705230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.716313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c985b4a-cb6f-42ff-8de4-d06379071e9f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.721325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.722470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.722470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.722470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.723133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.724734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.738175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1074/2131] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:14.742580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.755058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683037c9-6d70-41a2-9fa4-520f85051d9b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.760019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.760634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.760915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.761234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.761876 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.763822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc188c-a1d3-4f7d-aed8-1bfd19642dd5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.778734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1058/2132] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:14.780372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.792406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4c2c-9de5-49a1-9d9a-2e550c095c0c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:14.797138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:14.798322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.798606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.798928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] PATH_INFO: `/v3/projects/bf922510f54d4d40ac00486394d67a7a/users/74594b3982264446b73d14bad3f46045/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.815490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf922510f54d4d40ac00486394d67a7a, user_id=74594b3982264446b73d14bad3f46045, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:14.818178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89323d-f759-4c13-a817-aa89d662111a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:14.833780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1075/2133] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:14 2022] PUT /identity/v3/projects/bf922510f54d4d40ac00486394d67a7a/users/74594b3982264446b73d14bad3f46045/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:14.839012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.839347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.839698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.860984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:14.862545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:14.921104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee9fa8-41a1-4ea2-90e0-4242279dff46 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.921784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee9fa8-41a1-4ea2-90e0-4242279dff46 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.922885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee9fa8-41a1-4ea2-90e0-4242279dff46 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.925310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1076/2134] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:41:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:41:14.951411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1059/2135] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:14.958065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:14.958434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:14.958805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:14.980254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:14.981624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:14.996113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:14.997978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:15.053158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:15.093444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1077/2136] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:15.112127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:15.113886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:15.120334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a599-ec65-4f30-bea6-08db8adab348 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:15.124747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:15.127388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:15.128258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:15.128796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:15.132941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7f7a0-7b19-4eaa-949c-7fb5e99cd9d6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:15.142769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:15.143910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:15.144334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:15.144483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:15.144892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:15.147219 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79632de1-5d72-43cd-be1e-0b93cd9a7f26 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:15.155648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:15.159419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe08ad-8110-406c-872d-e5a66cd5f274 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:15.173737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1078/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:15.184722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1060/2138] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:15.963967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:15.964311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:15.964525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:15.998194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e1667-0ed4-48fa-9d8d-3bc314df3e74 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:15.998794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e1667-0ed4-48fa-9d8d-3bc314df3e74 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:15.999489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e1667-0ed4-48fa-9d8d-3bc314df3e74 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.003659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1061/2139] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:41:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:41:16.006263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] MFA Rules not processed for user `74594b3982264446b73d14bad3f46045`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:16.008197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.022398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.023006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.023515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.073090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:16.077134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.100977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1062/2140] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:41:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:16.111573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:16.113592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.130772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1079/2141] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:16.130772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:16.131024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bca2a-8d27-4376-b49c-a474e1353f7b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:16.135404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:16.136400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.136753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.137100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] PATH_INFO: `/v3/users/8c1d30d457d74c4e870eba4e2eb9384f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.138329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1d30d457d74c4e870eba4e2eb9384f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:16.140439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:16.157923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99baee-38c7-4b10-9524-dfdbb289bb70 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:16.167181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:16.168149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.168810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.169270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.174095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:16.183136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:16.185543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:16.201884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1d30d457d74c4e870eba4e2eb9384f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.202724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1d30d457d74c4e870eba4e2eb9384f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.210911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1080/2142] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:16.211462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1d30d457d74c4e870eba4e2eb9384f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.211742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:16.212356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1d30d457d74c4e870eba4e2eb9384f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.214615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.227618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1d30d457d74c4e870eba4e2eb9384f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.246963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1063/2143] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:16 2022] DELETE /identity/v3/users/8c1d30d457d74c4e870eba4e2eb9384f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:16.293538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb4bb-12e4-46d8-b77d-f1a2e9d55243 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:16.375706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:16.376362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.376631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.376903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.378668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.396004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:16.475711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:16.477553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:16.499786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:16.501973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:16.503447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1081/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:16.609271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5f54-35b2-4a00-9c2d-b9367cd4a12b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:16.622415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:16.623215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.623569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.624838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] PATH_INFO: `/v3/projects/1fd21d3eafe145049162fdee48446f2d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.627460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd21d3eafe145049162fdee48446f2d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:16.629401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:16.774334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd21d3eafe145049162fdee48446f2d', 'request_id': 'req-050c151f-ba30-45c7-9cb3-dacbc1277aea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.785798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd21d3eafe145049162fdee48446f2d', 'request_id': 'req-050c151f-ba30-45c7-9cb3-dacbc1277aea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.795467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd21d3eafe145049162fdee48446f2d', 'request_id': 'req-050c151f-ba30-45c7-9cb3-dacbc1277aea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.815053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] The token cache is being invalidate because project 1fd21d3eafe145049162fdee48446f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:16.816628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.820457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.825973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1064/2145] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:16 2022] DELETE /identity/v3/projects/1fd21d3eafe145049162fdee48446f2d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:16.826927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:16.839574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5aa88-64c1-4747-8ec5-c406179d5d7c service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:16.904179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:16.904886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:16.905339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:16.905451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] PATH_INFO: `/v3/users/e59dca975da9490db6c2d1004f5ba17a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:16.907353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59dca975da9490db6c2d1004f5ba17a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:16.909456 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:16.963830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59dca975da9490db6c2d1004f5ba17a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.964488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59dca975da9490db6c2d1004f5ba17a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.965496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59dca975da9490db6c2d1004f5ba17a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.987936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59dca975da9490db6c2d1004f5ba17a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:16.993442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0de06066-5d2c-4127-ae4c-8f167c38e6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59dca975da9490db6c2d1004f5ba17a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.023149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1082/2146] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:16 2022] DELETE /identity/v3/users/e59dca975da9490db6c2d1004f5ba17a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:17.270150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:17.271838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:17.287798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c151f-ba30-45c7-9cb3-dacbc1277aea admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:17.341037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:17.386949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:17.387636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.387930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.388187 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] PATH_INFO: `/v3/projects/1785aca198404e92af922a86497b1275` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.389987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1785aca198404e92af922a86497b1275)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:17.392955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:17.462923 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1785aca198404e92af922a86497b1275', 'request_id': 'req-643d928e-13d1-48db-a79a-326e4484f349'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.476179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1785aca198404e92af922a86497b1275', 'request_id': 'req-643d928e-13d1-48db-a79a-326e4484f349'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.489541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1785aca198404e92af922a86497b1275', 'request_id': 'req-643d928e-13d1-48db-a79a-326e4484f349'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.504988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] The token cache is being invalidate because project 1785aca198404e92af922a86497b1275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:17.505523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.507249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-643d928e-13d1-48db-a79a-326e4484f349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:17.511256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1065/2147] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:17 2022] DELETE /identity/v3/projects/1785aca198404e92af922a86497b1275 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:17.515816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.517074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.517478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.541837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:17.543315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:17.597106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:17.642847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1083/2148] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:17.657197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.657479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.657756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.695297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:17.696597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:17.706803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1066/2149] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:17.710799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:17.723278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fd0c5-2454-4323-9f98-f8bf52803d80 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:17.738957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:17.739592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.740074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.740330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.740983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:17.741451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:17.743499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:17.762740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1084/2150] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:17.766380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:17.776374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fba5a-5a68-4dda-bc29-1fe83ea27f63 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:17.782781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:17.783579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.783901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.784179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.784880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:17.787217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:17.846120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1067/2151] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:17 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:17.851204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:17.879029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba38530-76be-460b-a7f0-36c3d0ffd34a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:17.979578 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:17.980269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:17.980644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:17.980994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:17.981641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:17.983665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.017865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:18.024202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1085/2152] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:17 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:18.045153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceb097-3667-4093-b444-a968bce19953 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:18.051583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:18.052623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.052920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.053198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.054200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:18.056364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809087dc-e3bd-4fc8-ba03-cd6cc2db0f45 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.084301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1068/2153] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:18.085708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:18.116182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f1745-2f8e-44a0-bffe-a6ad74bd1a6f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:18.133551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:18.133551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.133551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.133551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.133866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:18.139962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.156354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1086/2154] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:18.159221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.159510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.159784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.169280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:18.191816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bd394-c252-4eaa-89fa-0306d4f5c000 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:18.193947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] MFA Rules not processed for user `74594b3982264446b73d14bad3f46045`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:18.195523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:18.213459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:18.215549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.216498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.217505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] PATH_INFO: `/v3/projects/3162b872e11943b48c0795ea99a42f4e/users/d5785cd1e41a4915832c030cb2848a8f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.264837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3162b872e11943b48c0795ea99a42f4e, user_id=d5785cd1e41a4915832c030cb2848a8f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:18.274164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10660a7c-27cb-4479-8eea-5f498c582ec7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.314956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1087/2155] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:18 2022] PUT /identity/v3/projects/3162b872e11943b48c0795ea99a42f4e/users/d5785cd1e41a4915832c030cb2848a8f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:18.321194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.321879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.322559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.334742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1069/2156] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:18.400666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] MFA Rules not processed for user `d5785cd1e41a4915832c030cb2848a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:18.402023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:18.564834 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1088/2157] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:18.569792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:18.584080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f2821-19a9-4887-92ba-e780070616f2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:18.647624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:18.709556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:18.710338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.710775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.711377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.712406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:18.715692 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.764573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1070/2158] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:18 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:18.768853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:18.788730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75c31-3b42-457a-8e6d-729e7da7ddd2 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:18.795103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:18.940951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:18.942963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:18.943914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:18.943914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:18.944704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:18.947564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:18.983069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1089/2159] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:18 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:18.986880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.010283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f64a0-8244-4562-8558-2954b90b79e4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.018990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.020284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.020638 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.020978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.022226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.029652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.036975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1071/2160] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:19.040785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.065573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6260ffa-3d01-43ab-a584-25fcaeabc970 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.091609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.092534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.092817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.093171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.093840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.096038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f60a0-e30b-4530-a660-f3421899b892 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.114804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1090/2161] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:19.118403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.142190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e032e0-bcbb-49f3-a73b-552f21c3985f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.155706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.156384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.156697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.158809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] PATH_INFO: `/v3/projects/de88ec0ca0154f34af257c76c6dec3bf/users/be3611e3a811448bb842a61ba001a46d/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.179388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de88ec0ca0154f34af257c76c6dec3bf, user_id=be3611e3a811448bb842a61ba001a46d, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.182002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.208613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1072/2162] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:19 2022] PUT /identity/v3/projects/de88ec0ca0154f34af257c76c6dec3bf/users/be3611e3a811448bb842a61ba001a46d/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:19.220633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.220914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.221195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.280020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] MFA Rules not processed for user `be3611e3a811448bb842a61ba001a46d`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:19.281619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:19.382857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1091/2163] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:19.384512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.455375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.456860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:19.457084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c29a2-b1af-45e9-b820-16ad91c5ba20 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.462230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.463049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.463321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.463596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.465052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.467631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.472612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc648b6-8f49-4ab4-8d45-6fdc56bbfd0e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.477958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.478589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.478897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.479167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] PATH_INFO: `/v3/users/62897a3ee86d4dbbae38e801ad31ea07` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.480308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62897a3ee86d4dbbae38e801ad31ea07)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.482224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.501198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1073/2164] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:19 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:19.502300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.530814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62897a3ee86d4dbbae38e801ad31ea07'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.532168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62897a3ee86d4dbbae38e801ad31ea07'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.533043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62897a3ee86d4dbbae38e801ad31ea07'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.554455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62897a3ee86d4dbbae38e801ad31ea07'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.559220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:19.565020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62897a3ee86d4dbbae38e801ad31ea07'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.582034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1092/2165] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:19 2022] DELETE /identity/v3/users/62897a3ee86d4dbbae38e801ad31ea07 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:19.593627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65f5ae-f30a-4f05-a3e3-4e567cdec72a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.604580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:19.660040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.660659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.660939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.661265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.661902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.663965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.700074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1074/2166] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:19 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:19.703946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.705727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:19.721919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86eaca-5422-40c6-9832-3a651acc6462 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.733326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.734235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.734620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.735569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.736302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.738063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.748786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1093/2167] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:19.753275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.763672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aa38-2c52-4c12-b81f-dd92a4866a8c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.769416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.770075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.770377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.770616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] PATH_INFO: `/v3/projects/89b978a4231e48f79b2bc2b1a7394022/users/b7b5f69030c9484cbe89059869c5493a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.788635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.790526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:19.792536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b978a4231e48f79b2bc2b1a7394022, user_id=b7b5f69030c9484cbe89059869c5493a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.796872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.806436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb031d4-76b5-4a77-9ad3-e1cde82203f5 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.812437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:19.813285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.813644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.813990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] PATH_INFO: `/v3/projects/7ca17bd7984241b1bbca7854ceb9440b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.815837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca17bd7984241b1bbca7854ceb9440b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:19.822789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:19.822950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1075/2168] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:19 2022] PUT /identity/v3/projects/89b978a4231e48f79b2bc2b1a7394022/users/b7b5f69030c9484cbe89059869c5493a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:19.827790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:19.886572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca17bd7984241b1bbca7854ceb9440b', 'request_id': 'req-ea51926c-c2e2-478f-abc0-0d9818eec338'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.895558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca17bd7984241b1bbca7854ceb9440b', 'request_id': 'req-ea51926c-c2e2-478f-abc0-0d9818eec338'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.905476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca17bd7984241b1bbca7854ceb9440b', 'request_id': 'req-ea51926c-c2e2-478f-abc0-0d9818eec338'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.917406 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214fbed-b3d6-4227-b424-ff4e6f6b289f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:19.925349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] The token cache is being invalidate because project 7ca17bd7984241b1bbca7854ceb9440b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:19.925767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.926730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-ea51926c-c2e2-478f-abc0-0d9818eec338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:19.929401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1094/2169] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:19 2022] DELETE /identity/v3/projects/7ca17bd7984241b1bbca7854ceb9440b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:19.933974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490918-beaa-457e-802f-dc0b7e1e7bea None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:19.934263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490918-beaa-457e-802f-dc0b7e1e7bea None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:19.935307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490918-beaa-457e-802f-dc0b7e1e7bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:19.958264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c6490918-beaa-457e-802f-dc0b7e1e7bea None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:19.960382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c6490918-beaa-457e-802f-dc0b7e1e7bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.018504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.019161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.019468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.020103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.020656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.022254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.033118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1076/2170] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:20.037218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.038990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.052135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486abdb-51da-4236-90fa-84e8f31a065b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.054622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1095/2171] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:20.057873 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.058840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.059659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.060778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] PATH_INFO: `/v3/projects/89b978a4231e48f79b2bc2b1a7394022/users/b7b5f69030c9484cbe89059869c5493a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.063474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b978a4231e48f79b2bc2b1a7394022, user_id=b7b5f69030c9484cbe89059869c5493a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.063850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.064284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.064424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.066547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350741cd-ac9e-470e-afaf-eecb59056875 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.079178 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1077/2172] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:20 2022] PUT /identity/v3/projects/89b978a4231e48f79b2bc2b1a7394022/users/b7b5f69030c9484cbe89059869c5493a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:20.089193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.089484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.089757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.091076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:20.092302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.111662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:20.124192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] MFA Rules not processed for user `b7b5f69030c9484cbe89059869c5493a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:20.126591 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.154666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:20.169256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1096/2173] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:20.173244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.183511 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dd30d-6f27-4dd1-84ad-9bf84ff33c69 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.189069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.189609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.189897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.190174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.191159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:20.191302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.192934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.208219 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1097/2174] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:20.212833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.223247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65772-5df7-4214-817f-1603c94763c5 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.227989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.228592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.228872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.229148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.229774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.231840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.234819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1078/2175] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:20.246489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.248081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.263353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1098/2176] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:20 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:20.268129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.321977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dff31-7132-4dca-87c8-3c1a25139dd9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.343693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6ce6-a857-40b2-916d-885f06968e47 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.348878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.349579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.349763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.350040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.350671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.352639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.377484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1099/2177] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:20 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:20.382153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.384415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.385258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.385814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.385912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.387713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.393444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18ace-d43f-4d2e-98d0-9dc45a301e75 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.398784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.399393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.399674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.399946 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.400594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.402185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.417464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1100/2178] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:20.421503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.434018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769606-51d6-4d94-8982-0a42b6d3a21e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.440911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.440911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.441232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.441719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.442379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.444249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.472876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1101/2179] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:20.476882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.480167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.482485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d3c3-4f57-4d19-b18d-50aa9ba51433 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.488310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520c699-4cff-4335-a726-d3258ae95bd2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.493452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.494840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.495119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.495520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] PATH_INFO: `/v3/projects/af5acb73124c485aa87e60b4e5d4f6d4/users/f594c9344b2c472ba33ceef87ec88a0b/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.501826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1079/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:20.516755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5acb73124c485aa87e60b4e5d4f6d4, user_id=f594c9344b2c472ba33ceef87ec88a0b, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.520403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.543353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.543353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.543353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.546579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1102/2181] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:20 2022] PUT /identity/v3/projects/af5acb73124c485aa87e60b4e5d4f6d4/users/f594c9344b2c472ba33ceef87ec88a0b/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:20.555788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:20.582633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:20.585003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.669457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1080/2182] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:20.678430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.680712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.747989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988e29-ba00-4c4c-b0cc-e14b1de9cda9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.753078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.753699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.754465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.754465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.769931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.771915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65c5ac-44d7-41f6-9d82-9c733fe704ca service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.785021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1103/2183] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:20.821966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:20.823501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:20.838980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf135-4693-4c72-94ba-b0fe1c62d2dd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:20.845000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:20.845819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:20.846175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:20.846479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] PATH_INFO: `/v3/users/6f649a4547d743a788da2020f8cc768b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:20.847929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f649a4547d743a788da2020f8cc768b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:20.849977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:20.934001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f649a4547d743a788da2020f8cc768b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:20.934851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f649a4547d743a788da2020f8cc768b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:20.948426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f649a4547d743a788da2020f8cc768b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:20.952192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f649a4547d743a788da2020f8cc768b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:20.983137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f649a4547d743a788da2020f8cc768b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.020836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1081/2184] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:20 2022] DELETE /identity/v3/users/6f649a4547d743a788da2020f8cc768b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:21.025693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:21.026023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:21.026290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:21.058901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:21.060844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:21.088391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:21.183088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1104/2185] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:21.196164 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:21.198066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:21.289038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55027885-cfe5-426c-b2be-d97adb838849 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:21.294362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:21.295148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:21.295429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:21.295702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:21.306341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:21.308037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e019314-e90a-44e9-a0ad-052573ed22d1 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:21.319459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1082/2186] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:21.700201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:21.702487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:21.721183 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c571d95-778e-435e-8ef8-fad0ca0814e5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:21.733283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:21.734294 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:21.734847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:21.735778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] PATH_INFO: `/v3/projects/9fe88de137a54426a53c679c6d093bb1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:21.737131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe88de137a54426a53c679c6d093bb1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:21.740498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:21.844916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe88de137a54426a53c679c6d093bb1', 'request_id': 'req-956eb59d-fca1-4a98-a465-e487bbd28458'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.878067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe88de137a54426a53c679c6d093bb1', 'request_id': 'req-956eb59d-fca1-4a98-a465-e487bbd28458'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.896460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe88de137a54426a53c679c6d093bb1', 'request_id': 'req-956eb59d-fca1-4a98-a465-e487bbd28458'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.926475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] The token cache is being invalidate because project 9fe88de137a54426a53c679c6d093bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:21.926799 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.931128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-956eb59d-fca1-4a98-a465-e487bbd28458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:21.933526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1105/2187] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:21 2022] DELETE /identity/v3/projects/9fe88de137a54426a53c679c6d093bb1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:21.938276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:21.938803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:21.939024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:21.971650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:21.973691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:22.018261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:22.089801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1083/2188] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:22.100648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.101627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.101916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.137510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:22.139052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:22.160682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1106/2189] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:22.169826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.182239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acd8dc-a008-4738-b670-0baab32bb9d0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.188378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.189135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.189418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.189694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.190381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:22.190686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.192896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.215741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1084/2190] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:22.218287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.237717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e371d-b231-4174-911e-83c1151ca5b9 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.248253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.248876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.249153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.249640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.250058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.258822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.301405 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1107/2191] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:22.306595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.323440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a773-c4c2-437f-aad3-c7b514f74a7b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.336254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.339667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:22.409892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.410857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.411341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.411808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.412690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.414902 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.430017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c1128-edc1-4aa1-b721-8c74725e1bcf admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.435255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.436012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.436354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.436844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.439979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:22.443127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1085/2192] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:22 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:22.447490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.461149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc15b1-b400-43f7-90ab-f550fbeec997 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.467520 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.468720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.469066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.469519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.470728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.471233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:22.472819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.491560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1086/2193] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:22.494237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.515424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a5fcf-0c7f-48bd-9cce-cfa1b5ce4d4a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.527240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.528269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.528536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.529420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.530067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.538468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.547056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.550078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd784f-0ac1-48df-baec-96cd7022c968 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.552490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1087/2194] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:22.557024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.563947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1108/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:22.569421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06363b4-78b2-498f-af92-973a89d573da admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:22.575994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:22.576877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.577805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.578477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] PATH_INFO: `/v3/projects/8ecb670832ad450d8f078142252d7854/users/88d95ff83cdd49578b8c413804dbe006/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.601539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecb670832ad450d8f078142252d7854, user_id=88d95ff83cdd49578b8c413804dbe006, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:22.605847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:22.627708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1088/2196] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:22 2022] PUT /identity/v3/projects/8ecb670832ad450d8f078142252d7854/users/88d95ff83cdd49578b8c413804dbe006/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:22.642516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:22.642932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:22.643240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:22.684608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] MFA Rules not processed for user `88d95ff83cdd49578b8c413804dbe006`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:22.686197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:22.810271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1109/2197] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:22.815211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:22.983935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:23.010325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3ab-6777-4f5c-9410-cdf91a6c70c8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.022189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.023053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.023715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.025122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.025806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.028179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.065443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1089/2198] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:23.071212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.101721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f7547-6f34-4c87-b338-2e16024064a8 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.221883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.222259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.222540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.222860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.223562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.225745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.268783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1110/2199] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:23 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:23.273612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.285795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9386ac2-3cb3-4f01-ba6e-7c38a3636439 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.293688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.294872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.295324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.295706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.296521 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.298720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.314420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1090/2200] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:23.319084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.333224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c39c-36b1-4283-a55e-f4a8a7465b2b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.338537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.340257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.340891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.341385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] PATH_INFO: `/v3/projects/05603d4fbfe24c5e83c6b044d5262f47/users/d39d35b1d5454a4eb1d817c6d750d800/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.367009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05603d4fbfe24c5e83c6b044d5262f47, user_id=d39d35b1d5454a4eb1d817c6d750d800, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.378129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.439262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1111/2201] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:23 2022] PUT /identity/v3/projects/05603d4fbfe24c5e83c6b044d5262f47/users/d39d35b1d5454a4eb1d817c6d750d800/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:23.444062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.456338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed94289-0d3e-4e1f-83bb-ff1eab4fe1c0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.565488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.566618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.570750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.570750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.570750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.570750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cf24-d3a1-4d05-a9b1-16ff4c2cfb17 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.611455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1091/2202] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:23.612131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.627006 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e050f2-186e-4f4b-917a-8945738d9ff1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.634367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.635135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.635441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.635697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] PATH_INFO: `/v3/projects/05603d4fbfe24c5e83c6b044d5262f47/users/d39d35b1d5454a4eb1d817c6d750d800/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.638916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05603d4fbfe24c5e83c6b044d5262f47, user_id=d39d35b1d5454a4eb1d817c6d750d800, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.640920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.658765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1112/2203] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:23 2022] PUT /identity/v3/projects/05603d4fbfe24c5e83c6b044d5262f47/users/d39d35b1d5454a4eb1d817c6d750d800/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:23.671903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.673356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.674218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.700181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.708907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:23.742749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] MFA Rules not processed for user `d39d35b1d5454a4eb1d817c6d750d800`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:23.753819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:23.785059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bb63-2df6-406d-a90a-c899c1579715 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.789842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.791285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.791816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.792238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.795747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:23.866493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1092/2204] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:23.883095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:23.887131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:23.897967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:23.900412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:23.904614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314bef6-4d51-443a-ba39-297adc0a3dc3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:23.910760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:23.911462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:23.911833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:23.912149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:23.915265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1113/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:23.976567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:24.004458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:24.006879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29aef-e138-46b7-9280-41c25feaac3d service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:24.025139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1093/2206] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:26.954055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:27.085856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b674-5eb4-4dad-8c81-e3ccdc1131a0 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:27.096876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:27.097881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:27.098197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:27.098543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] PATH_INFO: `/v3/users/88d95ff83cdd49578b8c413804dbe006` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:27.101979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d95ff83cdd49578b8c413804dbe006)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:27.108063 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:27.175069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d95ff83cdd49578b8c413804dbe006'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:27.175908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d95ff83cdd49578b8c413804dbe006'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:27.179331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d95ff83cdd49578b8c413804dbe006'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:27.190387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:27.212071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d95ff83cdd49578b8c413804dbe006'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:27.228726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d95ff83cdd49578b8c413804dbe006'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:27.259551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1114/2207] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:26 2022] DELETE /identity/v3/users/88d95ff83cdd49578b8c413804dbe006 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:27.264021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:27.264562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:27.264646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:27.296273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:27.298413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:27.394534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1094/2208] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:27.405418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:27.408493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:27.486225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b4a0a-de99-4fcb-9dab-2078b3b4fd8c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:27.490386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:27.491346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:27.491737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:27.492288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:27.504959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:27.506446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:27.518822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1115/2209] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:28.255001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:28.256762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.274090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267b2d2-faaa-4283-8377-5d3946b730cd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:28.289667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:28.290528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.291112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.291291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] PATH_INFO: `/v3/projects/8ecb670832ad450d8f078142252d7854` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:28.292813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ecb670832ad450d8f078142252d7854)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:28.294870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:28.392924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:28.394364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.408857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ecb670832ad450d8f078142252d7854', 'request_id': 'req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.415505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17df8-9c8c-4179-9895-2e860e233d07 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:28.425335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ecb670832ad450d8f078142252d7854', 'request_id': 'req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.445316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ecb670832ad450d8f078142252d7854', 'request_id': 'req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.458275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] The token cache is being invalidate because project 8ecb670832ad450d8f078142252d7854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:28.458568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.459280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.462745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1095/2210] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:28 2022] DELETE /identity/v3/projects/8ecb670832ad450d8f078142252d7854 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:28.464912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:28.468499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.492566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:28.493497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.493993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.494793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] PATH_INFO: `/v3/users/d5785cd1e41a4915832c030cb2848a8f` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:28.495840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5785cd1e41a4915832c030cb2848a8f)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:28.498938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:28.559056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe2ca-cbc0-4f4b-9030-d8289fefbb7d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:28.561326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5785cd1e41a4915832c030cb2848a8f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.561996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5785cd1e41a4915832c030cb2848a8f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.562633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5785cd1e41a4915832c030cb2848a8f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.564351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:28.565095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.565472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.565814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] PATH_INFO: `/v3/users/d39d35b1d5454a4eb1d817c6d750d800` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:28.567462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39d35b1d5454a4eb1d817c6d750d800)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:28.569631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:28.576122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5785cd1e41a4915832c030cb2848a8f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.582246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:28.589584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f6e673ea-c6cb-4508-81b9-a4690099be70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5785cd1e41a4915832c030cb2848a8f'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.597921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:28.613072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1116/2211] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:28 2022] DELETE /identity/v3/users/d5785cd1e41a4915832c030cb2848a8f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:28.617345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.617900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.618249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:28.650199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:28.651769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.664375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39d35b1d5454a4eb1d817c6d750d800'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.665280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39d35b1d5454a4eb1d817c6d750d800'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.675192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39d35b1d5454a4eb1d817c6d750d800'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.675789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39d35b1d5454a4eb1d817c6d750d800'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.695654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1465b58f-ad81-44be-9f1d-a782fc5d547b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39d35b1d5454a4eb1d817c6d750d800'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:28.715396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1096/2212] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:28 2022] DELETE /identity/v3/users/d39d35b1d5454a4eb1d817c6d750d800 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:28.715396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.715396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.715396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:28.756428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1117/2213] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:28.759838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] MFA Rules not processed for user `f594c9344b2c472ba33ceef87ec88a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:28.763084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.771900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:28.773682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.857696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570afe4-e3ae-4307-9fc7-28e66a5736d0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:28.880012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1097/2214] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:28.902132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:28.904402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:28.942396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:28.943404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:28.944090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:28.944429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.046833 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.050113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.060436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b86ada-2924-4993-b5e5-13d980b51def None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.067624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.068151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.068431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.069214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.070050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:29.083870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.083870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.099084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1118/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:29.100931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.102977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.118358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e11da-d6b7-4179-90bb-1e75dc206e92 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.126277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.127182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.127575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.127974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] PATH_INFO: `/v3/projects/05603d4fbfe24c5e83c6b044d5262f47` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.130058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=05603d4fbfe24c5e83c6b044d5262f47)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.133121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.135803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1098/2216] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:29.136661 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.146892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.162726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:29.177548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9a9d-4d2e-47bf-bb56-2f9ebdc5122f service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.198834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.201074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.202533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.202632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] PATH_INFO: `/v3/users/31f7af920dda4877a5fd64699ec37ee7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.206200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f7af920dda4877a5fd64699ec37ee7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.213390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.253127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05603d4fbfe24c5e83c6b044d5262f47', 'request_id': 'req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.285738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05603d4fbfe24c5e83c6b044d5262f47', 'request_id': 'req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.328190 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05603d4fbfe24c5e83c6b044d5262f47', 'request_id': 'req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.377703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] The token cache is being invalidate because project 05603d4fbfe24c5e83c6b044d5262f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:29.378032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.382319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2d868b86-c0cb-489f-8c95-9b9230ac5fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.388073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1119/2217] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:29 2022] DELETE /identity/v3/projects/05603d4fbfe24c5e83c6b044d5262f47 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:29.393030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f7af920dda4877a5fd64699ec37ee7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.393541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8d5b-2e10-4168-810f-c62580343bff None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.393898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f7af920dda4877a5fd64699ec37ee7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.393978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8d5b-2e10-4168-810f-c62580343bff None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.394071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8d5b-2e10-4168-810f-c62580343bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.408094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f7af920dda4877a5fd64699ec37ee7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.412270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f7af920dda4877a5fd64699ec37ee7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.425804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8bfe8d5b-2e10-4168-810f-c62580343bff None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:29.428119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8bfe8d5b-2e10-4168-810f-c62580343bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.440186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9162e01b-1f79-45b1-a509-023ad448aad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f7af920dda4877a5fd64699ec37ee7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:29.467434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1099/2218] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:29 2022] DELETE /identity/v3/users/31f7af920dda4877a5fd64699ec37ee7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:29.471868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.472150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.472436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.513891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:29.515240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.597650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1120/2219] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:29.598917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.599203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.599907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.629835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:29.631023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.705933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1100/2220] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:29.715058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.715829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1121/2221] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:29.727609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f5625-89af-4f5a-a933-2ffbe72bbd48 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.732374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.732985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.733300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.733528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.734220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:29.734710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.736967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.743400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.744893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.769303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1101/2222] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:29.781680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.807810 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de277-acc4-461e-a0f0-3e71746ecd35 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.822831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.823665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.824057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.824438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.825491 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.826552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:29.827833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.871697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d49279-f5a1-40fe-a275-8bb2416814da None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:29.882021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:29.884066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:29.884687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:29.884687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:29.903245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1102/2223] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:29 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:29.903874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.914653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:29.919756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:29.948486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1122/2224] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:29.949484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:29.950840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:29.966907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:29.967191 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f46a3-5190-416b-ba6c-43e31d976dd4 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.038215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f706014-a755-483f-8881-2d73db787ea1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.059805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.062024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.062956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.063258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.064183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.067395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.098587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1103/2225] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:29 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:30.102494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.104160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.104897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.105634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.105912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] PATH_INFO: `/v3/projects/3162b872e11943b48c0795ea99a42f4e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.107575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3162b872e11943b48c0795ea99a42f4e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.112288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.115977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7a1a-121e-4d64-9a3c-f16369100655 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.123777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.124633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.125198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.125388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.126777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.128816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.146514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1104/2226] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:30.149773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.166617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764364e-6c34-44e9-9414-84b89a746dfe admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.174865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.175502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.175936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.176404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.177494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.179976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.196173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1105/2227] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:30.200848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.214198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4c86c-5943-4fb2-ae44-4d3d5b1bd692 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.224795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3162b872e11943b48c0795ea99a42f4e', 'request_id': 'req-d26103d8-542a-40be-b937-efd126c4715d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.238135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3162b872e11943b48c0795ea99a42f4e', 'request_id': 'req-d26103d8-542a-40be-b937-efd126c4715d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.262034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3162b872e11943b48c0795ea99a42f4e', 'request_id': 'req-d26103d8-542a-40be-b937-efd126c4715d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.284287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] The token cache is being invalidate because project 3162b872e11943b48c0795ea99a42f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:30.284638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.286099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.288234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1123/2228] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:29 2022] DELETE /identity/v3/projects/3162b872e11943b48c0795ea99a42f4e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:30.294089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.296452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:30.336608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.339496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.339794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.340110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] PATH_INFO: `/v3/projects/71f10902cae64a09b587e9a481ce9e41/users/c7f3a776c48e4cebb7624d215f93cd1a/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.378207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:30.379972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f10902cae64a09b587e9a481ce9e41, user_id=c7f3a776c48e4cebb7624d215f93cd1a, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.384040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b7189-34a8-42c7-bbb4-e069c8956336 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.385218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26103d8-542a-40be-b937-efd126c4715d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.404960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.405626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.405904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.406178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] PATH_INFO: `/v3/users/be3611e3a811448bb842a61ba001a46d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.408372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3611e3a811448bb842a61ba001a46d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.409141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1106/2229] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:30 2022] PUT /identity/v3/projects/71f10902cae64a09b587e9a481ce9e41/users/c7f3a776c48e4cebb7624d215f93cd1a/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:30.411352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.414007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.415060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.415565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.444848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:30.447255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:30.492089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3611e3a811448bb842a61ba001a46d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.492936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3611e3a811448bb842a61ba001a46d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.494225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3611e3a811448bb842a61ba001a46d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.510037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3611e3a811448bb842a61ba001a46d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.523410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3611e3a811448bb842a61ba001a46d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.546568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1124/2230] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:30 2022] DELETE /identity/v3/users/be3611e3a811448bb842a61ba001a46d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:30.548797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1107/2231] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:30.557620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.559447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:30.562994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:30.565433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:30.657629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:30.716037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489ca6f-681f-439a-83a3-c0d08a03148c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.716956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136987ab-6f09-48b7-8dc7-b757eba7ab9c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:30.726382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.728048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.728665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.729110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] PATH_INFO: `/v3/projects/3744db727b864f949971cd8f70cb372a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.730813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3744db727b864f949971cd8f70cb372a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.734080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.826807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:30.827748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.828294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.828862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.844819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:30.847498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05048b9-0a8f-4a70-8e13-c1ff754a68f8 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:30.849547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3744db727b864f949971cd8f70cb372a', 'request_id': 'req-e44d07cb-08fc-4ed9-9bad-1df40412e808'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.870586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1108/2232] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:30.873803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3744db727b864f949971cd8f70cb372a', 'request_id': 'req-e44d07cb-08fc-4ed9-9bad-1df40412e808'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.884215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3744db727b864f949971cd8f70cb372a', 'request_id': 'req-e44d07cb-08fc-4ed9-9bad-1df40412e808'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.913540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] The token cache is being invalidate because project 3744db727b864f949971cd8f70cb372a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:30.913883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.915210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:30.923962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1125/2233] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:30 2022] DELETE /identity/v3/projects/3744db727b864f949971cd8f70cb372a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:30.926620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3002-dc58-40b7-8394-b1c8277595cf None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:30.927094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3002-dc58-40b7-8394-b1c8277595cf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:30.927569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3002-dc58-40b7-8394-b1c8277595cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:30.966674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2eca3002-dc58-40b7-8394-b1c8277595cf None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:30.971748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2eca3002-dc58-40b7-8394-b1c8277595cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.101570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.106069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.130809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:31.131984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1109/2234] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:31.143379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.144020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.144365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.230643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:31.234637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.258272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1110/2235] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:31.262295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.285346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48126a-a66e-4c6b-ab16-72eae8089c35 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.297140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.297414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.297682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.298448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.299326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:31.299633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.301480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.318328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1111/2236] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:41:31.330677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.332468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d07cb-08fc-4ed9-9bad-1df40412e808 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.339871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.340667 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.340939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.341892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] PATH_INFO: `/v3/projects/de88ec0ca0154f34af257c76c6dec3bf` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.343517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=de88ec0ca0154f34af257c76c6dec3bf)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.346097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.368562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d335e-9c84-4770-aa5c-10a9881e66f4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.384362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.385254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.385719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.386901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.388151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.392465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.445718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de88ec0ca0154f34af257c76c6dec3bf', 'request_id': 'req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.451746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1112/2237] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:31 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:31.454296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.470510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de88ec0ca0154f34af257c76c6dec3bf', 'request_id': 'req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.510877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de88ec0ca0154f34af257c76c6dec3bf', 'request_id': 'req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.528362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] The token cache is being invalidate because project de88ec0ca0154f34af257c76c6dec3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:31.528987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.529936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.537157 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1126/2238] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:31 2022] DELETE /identity/v3/projects/de88ec0ca0154f34af257c76c6dec3bf => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:31.538141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.539678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.592531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:31.637105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426778f8-fff7-4711-b66b-ba13aa27d431 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.652433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.653392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.653392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.653583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.654220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.658177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.677324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06cf4-0dfb-4cc4-9a7d-010162ce94ed admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.686140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.686954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.687308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.687592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] PATH_INFO: `/v3/users/b7b5f69030c9484cbe89059869c5493a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.689610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b5f69030c9484cbe89059869c5493a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.691939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.764596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:31.769292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1113/2239] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:31 2022] POST /identity/v3/users => generated 288 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:31.770264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.772294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.773227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b5f69030c9484cbe89059869c5493a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.776118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b5f69030c9484cbe89059869c5493a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.776981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b5f69030c9484cbe89059869c5493a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.787157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261370a0-02b3-4e48-bc5f-1d4e0c615cb9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.799891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.801016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b5f69030c9484cbe89059869c5493a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.801182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.801361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.801628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.802297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.804343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.818628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1114/2240] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:31.823128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:31.824375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b5f69030c9484cbe89059869c5493a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:31.824965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:31.852434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd5d2f-82de-462f-a1b3-788d09874162 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:31.860878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:31.861529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:31.861844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:31.862119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:31.862816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:31.864416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c85c-c59c-43ff-bb11-0e27638c1e21 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:31.866923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1127/2241] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:31 2022] DELETE /identity/v3/users/b7b5f69030c9484cbe89059869c5493a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:31.876546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1115/2242] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:31.881185 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:32.024986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663c75b-07b0-437a-b83b-09bbf58f4155 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:32.035068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:32.035652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.036181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.037007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] PATH_INFO: `/v3/projects/1a16abd063184d498be68713568eb5e8/users/960b75e1da10468aa6f073dc5a7a48ec/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.068414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a16abd063184d498be68713568eb5e8, user_id=960b75e1da10468aa6f073dc5a7a48ec, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:32.072652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:32.114746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1128/2243] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:31 2022] PUT /identity/v3/projects/1a16abd063184d498be68713568eb5e8/users/960b75e1da10468aa6f073dc5a7a48ec/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:32.116393 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.117324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.117741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.189966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] MFA Rules not processed for user `960b75e1da10468aa6f073dc5a7a48ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:32.196917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:32.347443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1116/2244] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:32.351620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:32.455388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:32.456970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:32.465326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d3d3-397f-4ed1-9a3b-3bcaca4249d7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:32.473115 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:32.474288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.474911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.475392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.476429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:32.479012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:32.481132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9920c-9953-49e3-bb53-ac8a3b4acb36 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:32.491468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:32.492125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.493701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.493701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] PATH_INFO: `/v3/projects/89b978a4231e48f79b2bc2b1a7394022` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.497999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b978a4231e48f79b2bc2b1a7394022)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:32.500005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:32.538278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1129/2245] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:32 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:32.543751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:32.621505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b978a4231e48f79b2bc2b1a7394022', 'request_id': 'req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:32.661293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:32.675279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b978a4231e48f79b2bc2b1a7394022', 'request_id': 'req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:32.687267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b978a4231e48f79b2bc2b1a7394022', 'request_id': 'req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:32.702198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] The token cache is being invalidate because project 89b978a4231e48f79b2bc2b1a7394022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:32.702490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:32.703224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b89d0c16-ab9f-4d33-9860-f12f4d4eea8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:32.709339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1117/2246] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:32 2022] DELETE /identity/v3/projects/89b978a4231e48f79b2bc2b1a7394022 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:32.710908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f6e-5668-4cb2-ada2-8a387c26912d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.711344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f6e-5668-4cb2-ada2-8a387c26912d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.711579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f6e-5668-4cb2-ada2-8a387c26912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.771123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0bd54f6e-5668-4cb2-ada2-8a387c26912d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:32.772865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0bd54f6e-5668-4cb2-ada2-8a387c26912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:32.798901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859cc01-39c8-47bb-8c92-ea6c59ad004d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:32.802105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:32.873950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1118/2247] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:32.877855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b930b55-0021-477e-8504-b20852254c9f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.878323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b930b55-0021-477e-8504-b20852254c9f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.878823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b930b55-0021-477e-8504-b20852254c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.908895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0b930b55-0021-477e-8504-b20852254c9f None None] MFA Rules not processed for user `f594c9344b2c472ba33ceef87ec88a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:32.910278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0b930b55-0021-477e-8504-b20852254c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:32.951074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:32.952301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:32.952686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:32.953009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:32.954418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:32.957198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9163f-6f4d-494e-bc27-af427144a633 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.005345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1130/2248] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:32 2022] POST /identity/v3/users => generated 288 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:33.008574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.008873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.009155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.015238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1119/2249] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:33.018462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.018651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.018991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.058495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:33.060117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:33.065398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] MFA Rules not processed for user `c7f3a776c48e4cebb7624d215f93cd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:33.067084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:33.093039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1120/2250] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:33.093940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.095790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:33.119753 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56991ae8-a9ba-4aa6-acae-d5dda4944977 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.124997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.126282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.126550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.126860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.127651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.129508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.146654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1121/2251] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:33.146654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.162153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df4a-0d58-40a4-8b39-8542720b5b65 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.168010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.168672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.169078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.169348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.173194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:33.173194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.174884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.204300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1131/2252] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:33.208576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1122/2253] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:33.208725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.208827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.218138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:33.240128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd74247-e9a0-4513-986b-cdb2dd401a48 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.265380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.266845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.267626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.268054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.268849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.270624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.295224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1132/2254] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:33.296873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.310290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3d10-59d5-4340-ac52-7337069282aa admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.325037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.325669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.325952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.326230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.326897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.328877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.351602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e95e-cbd7-43c3-bfd9-3e74837d9fd3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.358504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.359364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.359567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.359700 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.382726 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.384342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.400987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1123/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:33.401930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.410089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1133/2256] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:33 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:33.411217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.417242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7af8-aa3f-4614-9286-cc13614fd208 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.437531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:33.529637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:33.530200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b26d-5008-49cd-9fff-aef16b76933a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.542731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.545652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.547072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.548777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.551084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.555660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.566233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.568537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.568537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.569172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] PATH_INFO: `/v3/projects/5ca604a6476b4be1840285f4849c9264/users/b6cde5b25bf94d6d9eb69a32896aecca/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.612179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1134/2257] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:33 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:33.617334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.618473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca604a6476b4be1840285f4849c9264, user_id=b6cde5b25bf94d6d9eb69a32896aecca, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.640423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c26ac-e116-4bec-b8ae-5cc20373d870 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.653433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f727b-1f7c-439f-a10a-7c6c861939da admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.681852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1124/2258] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:33 2022] PUT /identity/v3/projects/5ca604a6476b4be1840285f4849c9264/users/b6cde5b25bf94d6d9eb69a32896aecca/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:33.685084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.685339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.685552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.687774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.688360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.689561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.690431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.691265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.698234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.721508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1135/2259] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:33.724485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.754973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30017cb7-f0cb-4770-a036-db36f743fade admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:33.764371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] MFA Rules not processed for user `b6cde5b25bf94d6d9eb69a32896aecca`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:33.765646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:33.896738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:33.897364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:33.897645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:33.897921 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:33.900640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:33.906346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:33.940879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1136/2260] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:33.942096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:33.987964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd5d02-8056-4d26-86dd-2b5f3069b34e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.016703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.017420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.018462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.018877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] PATH_INFO: `/v3/projects/ccb2df9322ae4a8faf84990261bb94eb/users/71adb5c997594b6dbb2bdcf76023b6d3/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.027993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1125/2261] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:34.033431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.055974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f30de5-4d3e-49d2-8c35-f541a17c49e7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.059568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb2df9322ae4a8faf84990261bb94eb, user_id=71adb5c997594b6dbb2bdcf76023b6d3, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.062357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f6ebb-e702-46dc-9c22-4f8b82ab8d2c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.070047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.070733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.074606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.074994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.075671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.094760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.096308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1137/2262] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:33 2022] PUT /identity/v3/projects/ccb2df9322ae4a8faf84990261bb94eb/users/71adb5c997594b6dbb2bdcf76023b6d3/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:34.097934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.098216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.098489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.166952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1126/2263] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:34 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:34.171326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.175228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] MFA Rules not processed for user `71adb5c997594b6dbb2bdcf76023b6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:34.176612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:34.292399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:34.306739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1138/2264] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:34.320661 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.362735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745963f0-9055-4f49-9484-009f87b67a20 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.370021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.371075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.371467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.372012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.372988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.375117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.436719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1127/2265] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:34 2022] POST /identity/v3/users => generated 295 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:41:34.437879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.450739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279b08a-e933-496e-a0fc-f620aae322f0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.454603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.455282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.455573 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.455826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.461410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177d958-e29b-401c-a7fa-240569c6f584 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.465045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.467199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.470013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.471270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.472069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.472503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.473762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.476254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.488546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1139/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:34.495209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1128/2267] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:34.498635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.514821 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4322-6894-4766-9dd5-1778781703fb service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.524235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.525003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.525334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.525453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.526466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] PATH_INFO: `/v3/projects/ec750f662f914dd18f68c2613f204fd5/users/c8a208faadfb427dabc6bbb0d660acec/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.537962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97fa8-919c-4ccc-afd3-b611a7a773c5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.543564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.544342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.544755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.545125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] PATH_INFO: `/v3/users/71adb5c997594b6dbb2bdcf76023b6d3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.546434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71adb5c997594b6dbb2bdcf76023b6d3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.549125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec750f662f914dd18f68c2613f204fd5, user_id=c8a208faadfb427dabc6bbb0d660acec, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.549925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.553550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.568943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:34.573782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1140/2268] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:34 2022] PUT /identity/v3/projects/ec750f662f914dd18f68c2613f204fd5/users/c8a208faadfb427dabc6bbb0d660acec/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:34.579293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.650743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71adb5c997594b6dbb2bdcf76023b6d3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:34.651619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71adb5c997594b6dbb2bdcf76023b6d3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:34.659981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71adb5c997594b6dbb2bdcf76023b6d3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:34.660755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71adb5c997594b6dbb2bdcf76023b6d3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:34.672152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a2c78-1b0d-4d5b-9d8a-6dece509ebfb admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.675463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-48c34f94-3919-450f-b1d6-7b1cba2a8636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71adb5c997594b6dbb2bdcf76023b6d3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:34.679282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.679894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.680301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.680451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.681220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.682948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.692799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1129/2269] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:34 2022] DELETE /identity/v3/users/71adb5c997594b6dbb2bdcf76023b6d3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:34.697949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1141/2270] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:34.699420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.699751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.700205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.703171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.706929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:34.725970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909fbb-1ceb-4b7e-ae4b-df2ea7c5724d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.739020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:34.740568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:34.795125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:34.795773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.796052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.796327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] PATH_INFO: `/v3/projects/ec750f662f914dd18f68c2613f204fd5/users/c8a208faadfb427dabc6bbb0d660acec/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.798806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec750f662f914dd18f68c2613f204fd5, user_id=c8a208faadfb427dabc6bbb0d660acec, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:34.801760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f15a-e0c0-4456-9644-2222fe40d5fd admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:34.812608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1130/2271] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:34.818335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:34.819813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1142/2272] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:34 2022] PUT /identity/v3/projects/ec750f662f914dd18f68c2613f204fd5/users/c8a208faadfb427dabc6bbb0d660acec/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:34.822741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:34.832352 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:34.832886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:34.833343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:34.860018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:34.885627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] MFA Rules not processed for user `c8a208faadfb427dabc6bbb0d660acec`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:34.887399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:34.913426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4dc99-409e-4846-852f-41d61ac8a0b0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:34.974676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:34.988286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:35.000736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:35.001119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:35.001386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:35.034138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1143/2273] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:35.055517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:35.057156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:35.095204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:35.096857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967442e-2e16-458c-b672-aa6514a89419 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:35.113191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1131/2274] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:35.140624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff089e-87d7-40bc-9b39-5a1fcea16b8d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:35.145926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:35.148152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:35.149171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:35.149539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:35.153913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:35.284734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:35.286365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:35.338444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1144/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:41:35.369785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988ddf6-fad7-4fa0-8515-91087315896d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:35.370095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988ddf6-fad7-4fa0-8515-91087315896d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:35.370373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988ddf6-fad7-4fa0-8515-91087315896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:35.408462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5988ddf6-fad7-4fa0-8515-91087315896d None None] MFA Rules not processed for user `c7f3a776c48e4cebb7624d215f93cd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:35.410306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5988ddf6-fad7-4fa0-8515-91087315896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:35.562160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1132/2276] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:35.704164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:35.706887 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:35.723694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8ea8-63f2-431d-822c-b106265d7ecf service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:35.770633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:35.771472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:35.772004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:35.772818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] PATH_INFO: `/v3/projects/ccb2df9322ae4a8faf84990261bb94eb` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:35.774555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb2df9322ae4a8faf84990261bb94eb)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:35.779354 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:35.852203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:35.859447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb2df9322ae4a8faf84990261bb94eb', 'request_id': 'req-57ba870f-1058-4fcb-94ab-33999a09faa1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:35.874146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb2df9322ae4a8faf84990261bb94eb', 'request_id': 'req-57ba870f-1058-4fcb-94ab-33999a09faa1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:35.883135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb2df9322ae4a8faf84990261bb94eb', 'request_id': 'req-57ba870f-1058-4fcb-94ab-33999a09faa1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:35.901820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] The token cache is being invalidate because project ccb2df9322ae4a8faf84990261bb94eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:35.902112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:35.902848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-57ba870f-1058-4fcb-94ab-33999a09faa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:35.904626 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1145/2277] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:35 2022] DELETE /identity/v3/projects/ccb2df9322ae4a8faf84990261bb94eb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:35.909866 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68e71-0866-419c-b206-aee863880945 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:35.910182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68e71-0866-419c-b206-aee863880945 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:35.910463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68e71-0866-419c-b206-aee863880945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:35.932691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6ff68e71-0866-419c-b206-aee863880945 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:35.934310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ff68e71-0866-419c-b206-aee863880945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:35.997607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1133/2278] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:36.005356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.005720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.006002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.026451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:36.027687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:36.036242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1146/2279] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:36.040444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff68e71-0866-419c-b206-aee863880945 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.050777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff68e71-0866-419c-b206-aee863880945 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.056136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.056749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.057039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.057314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.058320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:36.058567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.060172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.089619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1134/2280] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:41:36.091538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.104507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87bf96-db0e-444c-a93f-b1139e607827 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.124310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.124956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.125244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.125548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.126182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.128265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.162658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1147/2281] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:36 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:36.167501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.184312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef218-f060-4baa-aebc-76869232ea43 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.217044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:36.286767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.287489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.287771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.288046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.288688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.290646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.324317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1135/2282] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:36 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:36.325658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.338393 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db32b7-6ab3-4ad8-82ca-96fba00285f1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.347492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.348276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.349514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.349907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.350720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.352422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.370107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1148/2283] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:36.370543 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.399479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6d22-455a-47eb-9546-efe9d7deb686 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.405151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.405925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.406212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.406562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.407877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.409720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62678653-2710-434c-b91d-7eed83a65e83 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.422201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1136/2284] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:36.425252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.440123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80bc6e-49ae-40f9-9019-c05f46aa1a3e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.446852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.447651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.447997 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.455138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] PATH_INFO: `/v3/projects/c48e8d8ba2df4be2ba414ece1e441587/users/6d79994ff95a46179dd636bb6e5563a2/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.474986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48e8d8ba2df4be2ba414ece1e441587, user_id=6d79994ff95a46179dd636bb6e5563a2, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.477972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.499478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1149/2285] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:36 2022] PUT /identity/v3/projects/c48e8d8ba2df4be2ba414ece1e441587/users/6d79994ff95a46179dd636bb6e5563a2/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:36.509443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.509738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.510020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.543203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] MFA Rules not processed for user `6d79994ff95a46179dd636bb6e5563a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:36.544451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:36.702643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1137/2286] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:36.720051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:36.723785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:36.879577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af5589-ceab-45b0-b63d-aa0212c30e04 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:36.896953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:36.898085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:36.898595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:36.899205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:36.913790 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:36.916201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:36.941901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1150/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:37.669907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:37.688894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede156-bf6d-4249-b9e6-e99e6a1804ae None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:37.693475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:37.694246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:37.703113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:37.703737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:37.706669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:37.847864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:37.849732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:37.886794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1138/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:38.876390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:38.890130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:38.906383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b94f39-15d3-4c33-9c67-020fdf209fd2 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:38.914246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:38.915576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:38.916194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:38.916782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:38.922329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:39.100703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:39.102178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b30b8-1a90-412b-981d-ec3c6d640f20 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:39.154734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1151/2289] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:43.249408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:43.250929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:43.297278 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:43.331093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5d7fa-f7bf-4956-b6bb-2075c37b632b service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:43.337751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:43.338408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:43.338743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:43.339027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] PATH_INFO: `/v3/users/960b75e1da10468aa6f073dc5a7a48ec` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:43.340471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=960b75e1da10468aa6f073dc5a7a48ec)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:43.342904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:43.412964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960b75e1da10468aa6f073dc5a7a48ec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:43.412964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960b75e1da10468aa6f073dc5a7a48ec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:43.424329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960b75e1da10468aa6f073dc5a7a48ec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:43.425104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960b75e1da10468aa6f073dc5a7a48ec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:43.441492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960b75e1da10468aa6f073dc5a7a48ec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:43.459150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1139/2290] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:43 2022] DELETE /identity/v3/users/960b75e1da10468aa6f073dc5a7a48ec => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:43.463935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:43.464435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:43.464659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:43.472314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:43.486048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:43.488249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:43.566487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1152/2291] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:41:43.576608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:43.578185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:43.660317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000619-8d96-4e10-a624-a00c015a3ba8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:43.664600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:43.665252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:43.665555 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:43.665912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:43.676420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:43.677951 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:43.691899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1140/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:44.264615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:44.267102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:44.280034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85f45-bfa5-4393-b112-0bf42ce201ce None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:44.285119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:44.285484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:44.286424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:44.288108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] PATH_INFO: `/v3/projects/1a16abd063184d498be68713568eb5e8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:44.289465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a16abd063184d498be68713568eb5e8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:44.293308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:44.353073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a16abd063184d498be68713568eb5e8', 'request_id': 'req-618e1226-d08f-4684-a6ff-1e0c7b5017b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.364928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a16abd063184d498be68713568eb5e8', 'request_id': 'req-618e1226-d08f-4684-a6ff-1e0c7b5017b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.372359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a16abd063184d498be68713568eb5e8', 'request_id': 'req-618e1226-d08f-4684-a6ff-1e0c7b5017b3'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.385631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] The token cache is being invalidate because project 1a16abd063184d498be68713568eb5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:44.386008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.386820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.388707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1153/2293] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:44 2022] DELETE /identity/v3/projects/1a16abd063184d498be68713568eb5e8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:44.393083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:44.403325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628067-3b15-41a3-b55a-ab0b0467fd55 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:44.477329 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:44.492942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:44.493603 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:44.493861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:44.494153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] PATH_INFO: `/v3/users/b6cde5b25bf94d6d9eb69a32896aecca` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:44.495274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6cde5b25bf94d6d9eb69a32896aecca)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:44.497185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:44.546139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6cde5b25bf94d6d9eb69a32896aecca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.546826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6cde5b25bf94d6d9eb69a32896aecca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.553675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6cde5b25bf94d6d9eb69a32896aecca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.554733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6cde5b25bf94d6d9eb69a32896aecca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.567420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6cde5b25bf94d6d9eb69a32896aecca'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:44.581992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1141/2294] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:44 2022] DELETE /identity/v3/users/b6cde5b25bf94d6d9eb69a32896aecca => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:44.998410 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:45.001991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:45.024826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e1226-d08f-4684-a6ff-1e0c7b5017b3 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:45.112058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:45.112696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:45.112975 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:45.113245 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] PATH_INFO: `/v3/projects/5ca604a6476b4be1840285f4849c9264` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:45.114488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca604a6476b4be1840285f4849c9264)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:45.116445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:45.150034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:45.172984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca604a6476b4be1840285f4849c9264', 'request_id': 'req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.180734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca604a6476b4be1840285f4849c9264', 'request_id': 'req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.189058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca604a6476b4be1840285f4849c9264', 'request_id': 'req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.202249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] The token cache is being invalidate because project 5ca604a6476b4be1840285f4849c9264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:45.202536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.203283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.205509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1154/2295] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:44 2022] DELETE /identity/v3/projects/5ca604a6476b4be1840285f4849c9264 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:45.209713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:45.211314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:45.224442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad71176-54da-4bec-a6c4-aa7899c87d77 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:45.294383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:45.295042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:45.295317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:45.295592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] PATH_INFO: `/v3/users/c8a208faadfb427dabc6bbb0d660acec` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:45.296841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a208faadfb427dabc6bbb0d660acec)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:45.300102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:45.367024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a208faadfb427dabc6bbb0d660acec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.367764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a208faadfb427dabc6bbb0d660acec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.374993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a208faadfb427dabc6bbb0d660acec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.376271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a208faadfb427dabc6bbb0d660acec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.392484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5916cf95-edb5-4788-ad78-3ef61134ac81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a208faadfb427dabc6bbb0d660acec'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.395463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:45.415503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1142/2296] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:45 2022] DELETE /identity/v3/users/c8a208faadfb427dabc6bbb0d660acec => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:41:45.823586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:45.825209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:45.847524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05406-4ffb-46e4-a4d4-4f41dae9557e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:45.912251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:45.912992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:45.913241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:45.913877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] PATH_INFO: `/v3/projects/ec750f662f914dd18f68c2613f204fd5` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:45.914770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec750f662f914dd18f68c2613f204fd5)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:45.916702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:45.966803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec750f662f914dd18f68c2613f204fd5', 'request_id': 'req-f5180f91-ecbf-4267-84ae-a9df35094253'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.976903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec750f662f914dd18f68c2613f204fd5', 'request_id': 'req-f5180f91-ecbf-4267-84ae-a9df35094253'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.984144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec750f662f914dd18f68c2613f204fd5', 'request_id': 'req-f5180f91-ecbf-4267-84ae-a9df35094253'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.997091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] The token cache is being invalidate because project ec750f662f914dd18f68c2613f204fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:45.997386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.998077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f5180f91-ecbf-4267-84ae-a9df35094253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:45.999692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1155/2297] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:45 2022] DELETE /identity/v3/projects/ec750f662f914dd18f68c2613f204fd5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:46.005277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.005686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.006087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.029431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:46.030768 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:46.102440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1143/2298] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:46.110993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.111343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.111627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.134934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:46.136268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:46.145060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1156/2299] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:46.148985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.163488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc49c-e91f-4cc1-9099-61a08876e4c6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.168358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.168973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.169255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.169744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.170403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:46.170733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.172391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.191832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1144/2300] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:41:46.193206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.203184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded576-11e7-4e67-bcce-7f9564db47e6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.208913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.209595 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.210082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.210551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.211342 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.213469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.244670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1157/2301] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:46 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:46.255874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.266918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3acfc8-49d5-4a5d-aa52-f1639bca0f0f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.325575 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.326177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.326451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.326749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.327361 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.329420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.350277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1145/2302] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:46 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:46.354347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.365726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76e02-25d7-44dd-a4e1-19b2fea626ec admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.370461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.371090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.371364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.371633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.372276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.373867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.384675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1158/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:41:46.388755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.398825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7c4ec-dd19-4c8c-8a1f-633083773aef admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.403608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.404205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.404488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.404765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.405409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.407542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed39980-f93e-4c50-a737-b4fd0ac6806f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.418449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1146/2304] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:46.423408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.434281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b7ac-f025-4e8b-8722-9ee09f48a1f1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.439485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.440148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.440425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.440700 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] PATH_INFO: `/v3/projects/b5d92f1b456144e5b104911b8188a8d1/users/cdce2cc8d0284f69b6d6b75a44a173a4/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.443156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:46.460530 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d92f1b456144e5b104911b8188a8d1, user_id=cdce2cc8d0284f69b6d6b75a44a173a4, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.463398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.477210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1159/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:46 2022] PUT /identity/v3/projects/b5d92f1b456144e5b104911b8188a8d1/users/cdce2cc8d0284f69b6d6b75a44a173a4/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:46.489005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.489293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.489570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.517398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] MFA Rules not processed for user `cdce2cc8d0284f69b6d6b75a44a173a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:46.518612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:46.568475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:46.593616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1147/2306] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:46.606072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:46.607912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:46.671642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423931-d093-4fbf-aa30-0f4300590083 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:46.676289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:46.676931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:46.677229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:46.677479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:46.686384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:46.687865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634d8dd-91a6-488f-83a9-37d42bead671 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:46.697499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1160/2307] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:47.924534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:47.925930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:47.995083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182900d-e267-4c97-8a16-5a8e4a99f6da None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:47.999750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:48.000505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:48.001287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:48.001561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] PATH_INFO: `/v3/users/6d79994ff95a46179dd636bb6e5563a2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:48.002740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d79994ff95a46179dd636bb6e5563a2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:48.004698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:48.057920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d79994ff95a46179dd636bb6e5563a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.061082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d79994ff95a46179dd636bb6e5563a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.069213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d79994ff95a46179dd636bb6e5563a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.070005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d79994ff95a46179dd636bb6e5563a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.086823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d79994ff95a46179dd636bb6e5563a2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.100671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1148/2308] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:47 2022] DELETE /identity/v3/users/6d79994ff95a46179dd636bb6e5563a2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:48.108029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:48.108496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:48.108621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:48.129260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:48.130742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:48.210046 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1161/2309] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:48.224131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:48.225797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:48.360644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0adda-13f3-46fb-b5e2-afd3ffca9e7d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:48.364434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:48.365092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:48.365372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:48.366247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:48.377774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:48.379407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:48.390265 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1149/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:48.719900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:48.721450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:48.734789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0438b-b49d-4822-a035-9cc0f8bf134c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:48.747962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:48.748635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:48.748913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:48.749187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] PATH_INFO: `/v3/projects/c48e8d8ba2df4be2ba414ece1e441587` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:48.750434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48e8d8ba2df4be2ba414ece1e441587)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:48.753168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:48.815633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48e8d8ba2df4be2ba414ece1e441587', 'request_id': 'req-2b28167d-fde1-45b0-be34-3bab0edc2c3e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.823421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48e8d8ba2df4be2ba414ece1e441587', 'request_id': 'req-2b28167d-fde1-45b0-be34-3bab0edc2c3e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.830233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48e8d8ba2df4be2ba414ece1e441587', 'request_id': 'req-2b28167d-fde1-45b0-be34-3bab0edc2c3e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.840879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] The token cache is being invalidate because project c48e8d8ba2df4be2ba414ece1e441587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:48.841170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.841812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2b28167d-fde1-45b0-be34-3bab0edc2c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:48.843361 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1162/2311] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:48 2022] DELETE /identity/v3/projects/c48e8d8ba2df4be2ba414ece1e441587 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:49.437610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:49.440739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:49.453546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1418-a901-45b3-82d0-bfed25933071 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:49.466545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:49.535015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:49.535015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:49.535015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:49.535015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] PATH_INFO: `/v3/users/cdce2cc8d0284f69b6d6b75a44a173a4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:49.536703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdce2cc8d0284f69b6d6b75a44a173a4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:49.539418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:49.612213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdce2cc8d0284f69b6d6b75a44a173a4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:49.613138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdce2cc8d0284f69b6d6b75a44a173a4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:49.620814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdce2cc8d0284f69b6d6b75a44a173a4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:49.621834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdce2cc8d0284f69b6d6b75a44a173a4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:49.639649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdce2cc8d0284f69b6d6b75a44a173a4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:49.656557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1150/2312] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:41:49 2022] DELETE /identity/v3/users/cdce2cc8d0284f69b6d6b75a44a173a4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:49.661382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:49.661860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:49.662068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:49.684295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:49.685179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:49.689517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:49.781045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1163/2313] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:49.791590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:49.793039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:49.865504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d79691-c988-4d65-9d4b-7cf23ee1ab61 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:49.869960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:49.870713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:49.871091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:49.871442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:49.881908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:49.883594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e7d36-6083-459f-84ea-26957770b6fd service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:49.896343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1151/2314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:41:50.488003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:50.489919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:50.524859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d1abd-8685-4496-8d53-305abeb7803d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:50.529876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:50.531101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.531101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.531315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] PATH_INFO: `/v3/projects/b5d92f1b456144e5b104911b8188a8d1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.532788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d92f1b456144e5b104911b8188a8d1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:50.534261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:50.587905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d92f1b456144e5b104911b8188a8d1', 'request_id': 'req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:50.602350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d92f1b456144e5b104911b8188a8d1', 'request_id': 'req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:50.615114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d92f1b456144e5b104911b8188a8d1', 'request_id': 'req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:50.627033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] The token cache is being invalidate because project b5d92f1b456144e5b104911b8188a8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:41:50.627394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:50.628084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45a3dd36-d8b5-4213-a7af-4af62f1c5e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:41:50.630784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1164/2315] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:41:50 2022] DELETE /identity/v3/projects/b5d92f1b456144e5b104911b8188a8d1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:50.635719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.636228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.636578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.658490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:50.660064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:50.708062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:50.750843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1152/2316] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:50.761896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.761896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.761896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.783792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:50.785150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:50.793896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1165/2317] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:50.798010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:50.808788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138d4e-d8b8-437f-864e-0dc44262ee10 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:50.814351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:50.815011 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.815294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.815624 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.816486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:41:50.816794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:50.818640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:50.835616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1153/2318] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:41:50.836731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:50.851698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65117edc-dda4-4d39-ab29-60149355c2e0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:50.859858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:50.860544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.860854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.861074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.861731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:50.863710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:50.898741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1166/2319] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:41:50 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:50.900969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:50.911723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be925-4108-4cfe-a0ff-e641667a30e6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:50.971900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:50.972430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:50.972705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:50.972974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:50.973600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:50.975780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:50.998778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1154/2320] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:41:50 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:41:51.002429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:51.013202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86618ca1-741e-4fe4-bbd4-66161f005d3f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:51.018616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:51.021067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:51.021067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:51.021306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:51.021948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:51.023630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:51.034211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1167/2321] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:51.038471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:51.049165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dc08-9256-4450-93cf-9d85ccdbaf6a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:51.054096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:51.055025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:51.055607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:51.055887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:51.056558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:51.058201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4d39d-6d83-47dd-9f9d-4be971ce98c1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:51.073231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1155/2322] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:41:51.078392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:51.093784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66a750-c7b9-48d7-98af-24311cdabaf2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:51.100291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:51.101266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:51.101550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:51.101822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] PATH_INFO: `/v3/projects/21d17e9c953845b2b8a75644a726d1cf/users/a927fb6f30dc46dcba32a36e4811c5a2/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:51.122273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d17e9c953845b2b8a75644a726d1cf, user_id=a927fb6f30dc46dcba32a36e4811c5a2, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:51.123257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:51.138954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1168/2323] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:41:51 2022] PUT /identity/v3/projects/21d17e9c953845b2b8a75644a726d1cf/users/a927fb6f30dc46dcba32a36e4811c5a2/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:41:51.150072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd740bd-5bab-422b-8422-084bf0365c00 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:51.150518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd740bd-5bab-422b-8422-084bf0365c00 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:51.150859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd740bd-5bab-422b-8422-084bf0365c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:51.186494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8cd740bd-5bab-422b-8422-084bf0365c00 None None] MFA Rules not processed for user `a927fb6f30dc46dcba32a36e4811c5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:41:51.189149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8cd740bd-5bab-422b-8422-084bf0365c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:51.262293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1156/2324] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:41:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:41:53.987553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:41:53.989366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:41:54.022113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:41:54.049908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d515c-ca71-40bb-b935-b0fd832b4f8e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:41:54.053545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:41:54.054150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:41:54.054684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:41:54.054987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:41:54.063990 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:41:54.065470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:41:54.074584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1169/2325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:41:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:42:01.257675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:04.070339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=13 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:08.145052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5a148-ee25-47fc-97a1-123ea61991eb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:08.145536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5a148-ee25-47fc-97a1-123ea61991eb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:08.145681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5a148-ee25-47fc-97a1-123ea61991eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:08.166423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2de5a148-ee25-47fc-97a1-123ea61991eb None None] MFA Rules not processed for user `c7f3a776c48e4cebb7624d215f93cd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:08.167675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2de5a148-ee25-47fc-97a1-123ea61991eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:08.353745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1157/2326] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:18.344313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=25 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:29.439524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:29.501387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff6bcc-3a2a-4330-a968-05f242abc1a9 service glance] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:29.506005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:29.506633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:29.506944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:29.507217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] PATH_INFO: `/v3/users/a927fb6f30dc46dcba32a36e4811c5a2` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:29.508344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a927fb6f30dc46dcba32a36e4811c5a2)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:29.510257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:29.556475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a927fb6f30dc46dcba32a36e4811c5a2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:29.557117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a927fb6f30dc46dcba32a36e4811c5a2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:29.557718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a927fb6f30dc46dcba32a36e4811c5a2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:29.569401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a927fb6f30dc46dcba32a36e4811c5a2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:29.576670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a927fb6f30dc46dcba32a36e4811c5a2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:29.592560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1170/2327] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:42:29 2022] DELETE /identity/v3/users/a927fb6f30dc46dcba32a36e4811c5a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:29.597696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:29.598147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:29.598240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:29.618231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:29.619505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:29.677536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1158/2328] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:42:29.687525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:29.688952 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:29.747518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8b05-90bc-4530-bf2f-a20e78945632 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:29.751147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:29.751748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:29.752027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:29.752301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:29.761147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:29.762571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d12d6-18de-4979-8526-fd6e23ccb7d1 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:29.772360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1171/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:42:30.096818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.098401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:30.110867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87662e70-82e1-486b-b1f5-1c17d042c8b0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.115639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.116447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.116792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.117137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] PATH_INFO: `/v3/projects/21d17e9c953845b2b8a75644a726d1cf` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.118348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d17e9c953845b2b8a75644a726d1cf)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.120320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.167299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d17e9c953845b2b8a75644a726d1cf', 'request_id': 'req-c8652676-e19c-4903-a799-8487c2b34639'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:30.174159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d17e9c953845b2b8a75644a726d1cf', 'request_id': 'req-c8652676-e19c-4903-a799-8487c2b34639'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:30.180270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d17e9c953845b2b8a75644a726d1cf', 'request_id': 'req-c8652676-e19c-4903-a799-8487c2b34639'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:30.191110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] The token cache is being invalidate because project 21d17e9c953845b2b8a75644a726d1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:42:30.191399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:30.192026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8652676-e19c-4903-a799-8487c2b34639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:30.193544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1159/2330] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:42:30 2022] DELETE /identity/v3/projects/21d17e9c953845b2b8a75644a726d1cf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:30.199868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.200151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.200425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.220690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:30.221905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:30.239433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:30.284184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1172/2331] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:30.292128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.292415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.292723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.312447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:30.313662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:30.322257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1160/2332] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:30.326499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.336188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9d7c4-e192-46fc-8346-d91032c2fc98 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.340757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.341358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.341638 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.341909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.342558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:42:30.342891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.344490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.356419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1173/2333] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:42:30.360751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.371068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37838594-6aa7-46a7-8673-9ba6fcf6c92e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.375672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.376270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.376539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.376800 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.377428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.379962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.409874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1161/2334] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:42:30 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:42:30.413958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.424979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b9e1a-97fa-42f9-95a4-a5effa6dc1b6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.489248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.489876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.490166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.490374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.491044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.493013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.521955 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1174/2335] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:42:30 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:42:30.526062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.536561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60ddad-5e26-4f7e-8e3a-db96dc989300 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.541167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.541762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.542033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.542299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.542981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.544526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.555304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1162/2336] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:42:30.559630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.570715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e00cf1-dd03-48f6-960a-c24eff3c51ed admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.575789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.576387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.576827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.577025 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.577673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.579531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa039-d5a0-4f9e-a2f4-94162199507a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.591162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1175/2337] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:42:30.595496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.605416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aeae6-5874-41d1-ba25-5808d40f9310 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.609896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.610532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.610841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.611110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] PATH_INFO: `/v3/projects/e97fd16d61ef4d5ebc5f435ae8a18561/users/760115e8fb554582a016445f9aadc00c/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.626064 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97fd16d61ef4d5ebc5f435ae8a18561, user_id=760115e8fb554582a016445f9aadc00c, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.629388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.642649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1163/2338] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:42:30 2022] PUT /identity/v3/projects/e97fd16d61ef4d5ebc5f435ae8a18561/users/760115e8fb554582a016445f9aadc00c/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:30.648092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db160b8-d3eb-4975-8991-f6fae820a4a0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.648380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db160b8-d3eb-4975-8991-f6fae820a4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.648656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db160b8-d3eb-4975-8991-f6fae820a4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.669703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5db160b8-d3eb-4975-8991-f6fae820a4a0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:30.671031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5db160b8-d3eb-4975-8991-f6fae820a4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:30.738195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1176/2339] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:30.749756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:30.751453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:30.775248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:30.820422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5b8d-2ef1-4f1a-b363-b6ee9dd16f2a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:30.828048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:30.828709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:30.828982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:30.829252 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:30.849140 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:30.850573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:30.860495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1164/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:42:34.727574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:34.728096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:34.728202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:34.756243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] MFA Rules not processed for user `760115e8fb554582a016445f9aadc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:34.757399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:34.775009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:34.836248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1177/2341] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:34.840893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:34.853545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d2003-2dea-4d5f-890e-aa86745ae707 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:34.859949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:34.866525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:34.866836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:34.867248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:34.868069 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:34.871515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:34.902159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1165/2342] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:42:34 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:42:34.906136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:34.918447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c454-1427-42f9-a7e3-f6f760fca9b0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:35.003428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:35.004052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:35.004334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:35.004615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:35.005248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:35.007325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:35.029034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1178/2343] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:42:34 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:42:35.032972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:35.048144 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce219150-498a-4bb0-8a24-af91009f0ecb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:35.054009 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:35.054807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:35.055149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:35.055488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:35.056568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:35.058399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:35.094002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1166/2344] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:42:35.096441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:35.108582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d9a7a-0272-45f4-b1b9-ff786a2df2c8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:35.113794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:35.114463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:35.114869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:35.115114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] PATH_INFO: `/v3/projects/17df950364db43b393289f3f6c5175e1/users/676a3a75d1b74104b9d2387a7ddbb8bc/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:35.134069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17df950364db43b393289f3f6c5175e1, user_id=676a3a75d1b74104b9d2387a7ddbb8bc, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:35.136843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:35.151188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1179/2345] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:42:35 2022] PUT /identity/v3/projects/17df950364db43b393289f3f6c5175e1/users/676a3a75d1b74104b9d2387a7ddbb8bc/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:35.154950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:35.166228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e769d-26b1-4be5-875f-c0c20258e83c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:35.250872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:35.251644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:35.251971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:35.252320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:35.253174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:35.260861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47381e-7fd8-48c2-9ca9-8ad717610ba9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:35.273384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1167/2346] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:42:35.277330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:35.292114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92325e1-0ad2-480f-9112-05ae60c16e34 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:35.300717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:35.301645 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:35.302098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:35.302532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] PATH_INFO: `/v3/projects/17df950364db43b393289f3f6c5175e1/users/676a3a75d1b74104b9d2387a7ddbb8bc/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:35.306578 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17df950364db43b393289f3f6c5175e1, user_id=676a3a75d1b74104b9d2387a7ddbb8bc, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:35.309455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:35.331771 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1180/2347] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:42:35 2022] PUT /identity/v3/projects/17df950364db43b393289f3f6c5175e1/users/676a3a75d1b74104b9d2387a7ddbb8bc/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:40.380554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd2db-7ddd-4f6e-b190-d9847ea76141 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:40.380958 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd2db-7ddd-4f6e-b190-d9847ea76141 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:40.381271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd2db-7ddd-4f6e-b190-d9847ea76141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:40.441991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fbfcd2db-7ddd-4f6e-b190-d9847ea76141 None None] MFA Rules not processed for user `676a3a75d1b74104b9d2387a7ddbb8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:40.441991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fbfcd2db-7ddd-4f6e-b190-d9847ea76141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:40.560289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1168/2348] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:40.574072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:40.577311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:40.713553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef52c5d-e4ad-4b77-809c-d455035bfd0e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:40.719728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:40.720922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:40.721303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:40.722357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:40.730340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:40.849668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:40.852740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:40.864406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1181/2349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:42:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:42:43.267220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03a73-bb87-48ae-9292-b1e3c3b875de None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:43.267697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03a73-bb87-48ae-9292-b1e3c3b875de None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:43.267815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03a73-bb87-48ae-9292-b1e3c3b875de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:43.287261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-49d03a73-bb87-48ae-9292-b1e3c3b875de None None] MFA Rules not processed for user `ea609a2504e64f6f9380061080976a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:43.288473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-49d03a73-bb87-48ae-9292-b1e3c3b875de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:43.293733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:43.352838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1169/2350] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:50.858616 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=24 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:53.344796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=20 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:57.903551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:57.905473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:57.994055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3cd6c-3a58-4800-88db-f81f16617dc1 service nova] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:57.998662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:57.999365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:58.000247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:58.000550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] PATH_INFO: `/v3/users/934391322c1a4c53b7cb3d3a5dae3542` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:58.001713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=934391322c1a4c53b7cb3d3a5dae3542)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:58.004041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:58.081059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934391322c1a4c53b7cb3d3a5dae3542'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.082367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934391322c1a4c53b7cb3d3a5dae3542'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.083024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934391322c1a4c53b7cb3d3a5dae3542'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.097593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934391322c1a4c53b7cb3d3a5dae3542'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.107686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934391322c1a4c53b7cb3d3a5dae3542'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.122159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1182/2351] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:42:57 2022] DELETE /identity/v3/users/934391322c1a4c53b7cb3d3a5dae3542 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:58.126537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:58.127048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:58.127486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:58.152843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:58.153528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:58.227039 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1170/2352] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:58.237803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:58.239299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:58.380608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73df42-d413-4e05-b3da-08f8cf4b0e3c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:58.387768 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:58.388539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:58.388818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:58.389084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:58.399763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:58.401387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc933-b911-4da6-8467-bbc5fe17edc1 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:58.415791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1183/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:42:58.896369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:58.898524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:58.913802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd225-7a1e-43ea-8399-bac4083042e5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:58.918496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:58.919173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:58.919432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:58.919703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] PATH_INFO: `/v3/projects/0ad905c0f6ab4a79bca2d976c05d1c41` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:58.920845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad905c0f6ab4a79bca2d976c05d1c41)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:58.922786 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:58.981992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad905c0f6ab4a79bca2d976c05d1c41', 'request_id': 'req-efdd5c0a-c57c-4102-95db-cb9bfb06a003'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.989614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad905c0f6ab4a79bca2d976c05d1c41', 'request_id': 'req-efdd5c0a-c57c-4102-95db-cb9bfb06a003'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:58.996114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad905c0f6ab4a79bca2d976c05d1c41', 'request_id': 'req-efdd5c0a-c57c-4102-95db-cb9bfb06a003'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:59.007126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] The token cache is being invalidate because project 0ad905c0f6ab4a79bca2d976c05d1c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:42:59.007411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:59.008049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-efdd5c0a-c57c-4102-95db-cb9bfb06a003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:42:59.009541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1171/2354] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:42:58 2022] DELETE /identity/v3/projects/0ad905c0f6ab4a79bca2d976c05d1c41 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:59.014519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025871-cadd-4332-a44f-f721869c88db None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.015085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025871-cadd-4332-a44f-f721869c88db None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.015622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025871-cadd-4332-a44f-f721869c88db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.035263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ee025871-cadd-4332-a44f-f721869c88db None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:59.036502 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ee025871-cadd-4332-a44f-f721869c88db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:59.056818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:59.110021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1184/2355] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:59.119930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.120212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.120504 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.140764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:59.141966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:59.149950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1172/2356] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:42:59.154146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee025871-cadd-4332-a44f-f721869c88db None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.164375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee025871-cadd-4332-a44f-f721869c88db None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.169492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.170097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.170374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.170643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.171329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:42:59.171630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.173227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.186514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1185/2357] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:42:59.191132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.201490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ffe37-4c7e-4323-8db4-e4bfe8bf8bf5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.206919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.207524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.207797 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.208060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.208678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.210581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.239332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1173/2358] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:42:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:42:59.243843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.255623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09199733-f3c4-4b28-8636-d564d309b2e4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.328762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.329373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.329663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.329936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.330571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.332549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.353438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1186/2359] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:42:59 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:42:59.363397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.374437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7dad-fd7c-438c-b334-d20ba3e7eff5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.379351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.379952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.380262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.380539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.381189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.382818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.399468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.399893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1174/2360] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:42:59.413070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8888e-4fbe-4a7b-b39a-284123511735 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.417877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.418507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.419050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.419242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.419889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.421461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad863a30-ce3c-4bcf-89ab-620db6380192 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.433500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1187/2361] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:42:59.444528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.454505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645c5e-b55f-4ad4-a44c-e44c37c47cc5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.460596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.461257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.461536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.461809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] PATH_INFO: `/v3/projects/ea75360af6084fa0a228bb86e711482a/users/e7963a259502456da3f2113e08557a0f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.477791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea75360af6084fa0a228bb86e711482a, user_id=e7963a259502456da3f2113e08557a0f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.480505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.493895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1175/2362] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:42:59 2022] PUT /identity/v3/projects/ea75360af6084fa0a228bb86e711482a/users/e7963a259502456da3f2113e08557a0f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:42:59.504301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdce0b-f37e-47ae-a6cc-6563873b1d71 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.504548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdce0b-f37e-47ae-a6cc-6563873b1d71 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.504919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdce0b-f37e-47ae-a6cc-6563873b1d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.532810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b1cdce0b-f37e-47ae-a6cc-6563873b1d71 None None] MFA Rules not processed for user `e7963a259502456da3f2113e08557a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:42:59.534014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b1cdce0b-f37e-47ae-a6cc-6563873b1d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:59.613687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1188/2363] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:42:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:42:59.624363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:42:59.626080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:42:59.648071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:42:59.684508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750da240-2ce7-4a93-a24a-85c4be2d7d3f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:42:59.688188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:42:59.688807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:42:59.689080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:42:59.689348 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:42:59.697517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:42:59.698955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:42:59.707988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1176/2364] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:42:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:03.903291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c7e-f680-4eb4-9c55-d249aa7fd1cf None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:03.903797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c7e-f680-4eb4-9c55-d249aa7fd1cf None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:03.904763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8c7e-f680-4eb4-9c55-d249aa7fd1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:03.932458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-034e8c7e-f680-4eb4-9c55-d249aa7fd1cf None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:03.934224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-034e8c7e-f680-4eb4-9c55-d249aa7fd1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:03.948919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:04.005295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1189/2365] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:43:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:04.049305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:04.050834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:04.125229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36e86-918a-4c97-ba98-577a0300eeab service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:04.130063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:04.130656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:04.130967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:04.131235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:04.139272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:04.140845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7030-ffc7-4d3c-b5fe-6b00bb5c751c service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:04.149090 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1177/2366] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:04.253430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7106c8-098e-4d6b-aa31-696a3793934c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:04.254001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7106c8-098e-4d6b-aa31-696a3793934c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:04.254001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7106c8-098e-4d6b-aa31-696a3793934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:04.274485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0d7106c8-098e-4d6b-aa31-696a3793934c None None] MFA Rules not processed for user `e7963a259502456da3f2113e08557a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:04.275874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0d7106c8-098e-4d6b-aa31-696a3793934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:04.285044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1190/2367] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:10.475288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd42ef7-27c3-4838-b0ba-6859dbfa3d23 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:10.476148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd42ef7-27c3-4838-b0ba-6859dbfa3d23 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:10.476583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd42ef7-27c3-4838-b0ba-6859dbfa3d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:10.497230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fdd42ef7-27c3-4838-b0ba-6859dbfa3d23 None None] MFA Rules not processed for user `760115e8fb554582a016445f9aadc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:10.498620 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fdd42ef7-27c3-4838-b0ba-6859dbfa3d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:10.573277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1178/2368] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:43:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:12.603080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c869-a8ad-48d1-b325-642280088c7c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:12.603675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c869-a8ad-48d1-b325-642280088c7c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:12.603675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c869-a8ad-48d1-b325-642280088c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:12.633188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-df52c869-a8ad-48d1-b325-642280088c7c None None] MFA Rules not processed for user `f594c9344b2c472ba33ceef87ec88a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:12.647324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-df52c869-a8ad-48d1-b325-642280088c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:12.788485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1191/2369] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:43:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:20.564040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=74 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:22.775619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=45 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:28.165702 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:28.166234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:28.166234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:28.188698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] MFA Rules not processed for user `74594b3982264446b73d14bad3f46045`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:28.190031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:28.252295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:28.252897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:28.252897 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:28.259880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1179/2370] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:43:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:28.281901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:28.284242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:28.292948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1192/2371] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:43:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:28.305712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:28.318955 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb44f7-4f42-4760-b441-34134e0ee118 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:28.324356 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:28.324956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:28.325228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:28.325496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:28.334488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:28.336095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:28.344682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1180/2372] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:30.294498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:30.296598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:30.429862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46485376-716c-4923-8b4e-d1c1d1b41cc7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:30.437132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:30.437877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:30.438257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:30.438629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:30.441383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:30.463482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:30.465297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:30.483044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:30.485386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:30.501466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1193/2373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:30.605480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63740a-137b-42c4-991e-c1356b469534 service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:30.613751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:30.614576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:30.614893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:30.621231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] PATH_INFO: `/v3/users/c7f3a776c48e4cebb7624d215f93cd1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:30.622712 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f3a776c48e4cebb7624d215f93cd1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:30.626205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:30.700104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f3a776c48e4cebb7624d215f93cd1a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:30.701398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f3a776c48e4cebb7624d215f93cd1a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:30.727249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f3a776c48e4cebb7624d215f93cd1a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:30.727249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f3a776c48e4cebb7624d215f93cd1a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:30.764190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c3adf06a-e892-41b7-bdf6-de4bb452659a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f3a776c48e4cebb7624d215f93cd1a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:30.785742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1181/2374] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:43:30 2022] DELETE /identity/v3/users/c7f3a776c48e4cebb7624d215f93cd1a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:43:31.036094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:31.065579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868981-e115-4670-9676-5118fb537d88 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:31.194809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:31.195982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:31.195982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:31.196555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] PATH_INFO: `/v3/projects/71f10902cae64a09b587e9a481ce9e41` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:31.198403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f10902cae64a09b587e9a481ce9e41)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:31.200398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:31.279338 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f10902cae64a09b587e9a481ce9e41', 'request_id': 'req-958f49d9-0446-45d1-9f88-fde50c15d00d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:31.302567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f10902cae64a09b587e9a481ce9e41', 'request_id': 'req-958f49d9-0446-45d1-9f88-fde50c15d00d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:31.325939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f10902cae64a09b587e9a481ce9e41', 'request_id': 'req-958f49d9-0446-45d1-9f88-fde50c15d00d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:31.346221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] The token cache is being invalidate because project 71f10902cae64a09b587e9a481ce9e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:43:31.346926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:31.347816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:31.349850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1194/2375] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:43:31 2022] DELETE /identity/v3/projects/71f10902cae64a09b587e9a481ce9e41 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:43:32.923626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714c88-ab52-4a71-a982-6f1a4f7c1fb9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:32.924074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714c88-ab52-4a71-a982-6f1a4f7c1fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:32.924279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714c88-ab52-4a71-a982-6f1a4f7c1fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:32.953060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d7714c88-ab52-4a71-a982-6f1a4f7c1fb9 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:32.954342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d7714c88-ab52-4a71-a982-6f1a4f7c1fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:32.963224 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:33.032494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1182/2376] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:43:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:33.047144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:33.049728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:33.066237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:33.133525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f49d9-0446-45d1-9f88-fde50c15d00d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:33.138632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:33.139654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.140126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.140586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.154941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:33.154941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:33.160734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1195/2377] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:33.752453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf8e-783d-4f92-bb3f-f9de4483b6e2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.752755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf8e-783d-4f92-bb3f-f9de4483b6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.753004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf8e-783d-4f92-bb3f-f9de4483b6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.772063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-99d0bf8e-783d-4f92-bb3f-f9de4483b6e2 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:33.773260 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-99d0bf8e-783d-4f92-bb3f-f9de4483b6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:33.780320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1183/2378] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:43:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:33.793171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:33.804486 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb09818-8199-47c1-9398-d5a904c6da10 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:33.809827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:33.810420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.810716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.811003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.818923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:33.820538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:33.828550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1196/2379] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:34.088431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354d9e7-72b3-4cc9-844a-dafb61b86a4b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:34.088895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354d9e7-72b3-4cc9-844a-dafb61b86a4b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:34.088989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354d9e7-72b3-4cc9-844a-dafb61b86a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:34.110074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1354d9e7-72b3-4cc9-844a-dafb61b86a4b None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:34.111391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1354d9e7-72b3-4cc9-844a-dafb61b86a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:34.118566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1184/2380] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:34.130496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:34.141047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b835c7-87fa-4f58-87bf-49e638d7b128 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:34.147016 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:34.147669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:34.147947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:34.148240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:34.160882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:34.162851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:34.172961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1197/2381] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:34.514647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:34.514959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:34.515227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:34.535817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:34.536585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:34.544375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1185/2382] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:34.557675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:34.568711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262928c-346b-4c59-be42-a03321e95536 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:34.574297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:34.574932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:34.575211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:34.575482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:34.584005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:34.586158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f47d3-b3b9-45c0-a3f0-5dcfe0df9f0c service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:34.594519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1198/2383] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:43:36.768706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:36.770172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:36.847465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f4718-8436-48ec-8138-e6b5dc7ab096 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:36.852027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:36.852663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:36.852949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:36.853222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] PATH_INFO: `/v3/users/e7963a259502456da3f2113e08557a0f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:36.854392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7963a259502456da3f2113e08557a0f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:36.856362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:36.922040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7963a259502456da3f2113e08557a0f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:36.923368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7963a259502456da3f2113e08557a0f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:36.932461 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7963a259502456da3f2113e08557a0f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:36.933264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7963a259502456da3f2113e08557a0f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:36.950086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7963a259502456da3f2113e08557a0f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:36.971967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1186/2384] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:43:36 2022] DELETE /identity/v3/users/e7963a259502456da3f2113e08557a0f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:43:36.976775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:36.978122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:36.978558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:37.005382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:37.007048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:37.085719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1199/2385] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:37.096611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:37.098193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:37.128888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:37.185142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c024d-a9f8-4c38-afd6-3d30d3e370f4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:37.200711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:37.201346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:37.201627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:37.201903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:37.218979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:37.221037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:37.241898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1187/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:37.668503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:37.670044 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:37.687540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b802fa-6ff4-4dd5-bff4-fc3b08618937 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:37.693212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:37.698252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:37.698565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:37.698881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] PATH_INFO: `/v3/projects/ea75360af6084fa0a228bb86e711482a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:37.700135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea75360af6084fa0a228bb86e711482a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:37.702076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:37.759707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea75360af6084fa0a228bb86e711482a', 'request_id': 'req-90132b1d-3835-4853-97e7-79b16d93eac5'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:37.767831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea75360af6084fa0a228bb86e711482a', 'request_id': 'req-90132b1d-3835-4853-97e7-79b16d93eac5'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:37.769979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:37.777335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea75360af6084fa0a228bb86e711482a', 'request_id': 'req-90132b1d-3835-4853-97e7-79b16d93eac5'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:37.790136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] The token cache is being invalidate because project ea75360af6084fa0a228bb86e711482a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:43:37.790349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:37.791049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-90132b1d-3835-4853-97e7-79b16d93eac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:43:37.792552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1200/2387] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:43:37 2022] DELETE /identity/v3/projects/ea75360af6084fa0a228bb86e711482a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:43:44.047035 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:44.055963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f23d89-e7de-4acb-aba6-a8e62519e7af service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:44.104557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:44.105154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:44.105455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:44.105723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:44.107697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:44.169156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:44.170632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:44.180221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1188/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:47.786844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:54.175749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=69 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:26.028314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:26.028824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:26.028926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:26.053912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] MFA Rules not processed for user `ea609a2504e64f6f9380061080976a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:26.055794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:26.110536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1201/2389] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:33.086101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:33.087636 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:33.158720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9255-e7e1-48ec-b24b-4dc4463d6e32 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:33.163749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:33.164377 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.164659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.164934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] PATH_INFO: `/v3/users/74594b3982264446b73d14bad3f46045` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.166056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74594b3982264446b73d14bad3f46045)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:33.168007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:33.215728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74594b3982264446b73d14bad3f46045'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.216378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74594b3982264446b73d14bad3f46045'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.223850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74594b3982264446b73d14bad3f46045'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.224481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74594b3982264446b73d14bad3f46045'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.236541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ec5bf3f2-0f6c-4670-96f2-4c1f5e499d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74594b3982264446b73d14bad3f46045'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.249804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1189/2390] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:44:33 2022] DELETE /identity/v3/users/74594b3982264446b73d14bad3f46045 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:44:33.551399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:33.562426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff416-886c-4d18-973e-ef0145016fed None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:33.639382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:33.640031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.640311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.640586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] PATH_INFO: `/v3/projects/bf922510f54d4d40ac00486394d67a7a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.641746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf922510f54d4d40ac00486394d67a7a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:33.643730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:33.693300 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf922510f54d4d40ac00486394d67a7a', 'request_id': 'req-122e669b-1c5a-4b96-a7f9-7d665973f670'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.700644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf922510f54d4d40ac00486394d67a7a', 'request_id': 'req-122e669b-1c5a-4b96-a7f9-7d665973f670'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.706911 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf922510f54d4d40ac00486394d67a7a', 'request_id': 'req-122e669b-1c5a-4b96-a7f9-7d665973f670'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.717512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] The token cache is being invalidate because project bf922510f54d4d40ac00486394d67a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:44:33.717874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.718504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-122e669b-1c5a-4b96-a7f9-7d665973f670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:44:33.719992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1202/2391] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:44:33 2022] DELETE /identity/v3/projects/bf922510f54d4d40ac00486394d67a7a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:44:33.725100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.725382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.725654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.744615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:33.745796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:33.804773 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1190/2392] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:33.868326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.868592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.868873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.889309 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:33.890516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:33.898316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1203/2393] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:33.902424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:33.911950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40cb6f-145e-419d-9f82-60bd9420e120 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:33.916434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:33.917017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.917292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.917560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.918198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:44:33.918492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:33.920123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:33.932386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1191/2394] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:44:33.936485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:33.946212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728267ee-91d9-4443-90c5-9c4dcdc2bd53 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:33.950802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:33.951384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:33.951659 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:33.951927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:33.952551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:33.954422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:33.981220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1204/2395] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:44:33 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:44:33.985303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:33.995739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74aa4-d1a6-4754-8ea2-709296c66341 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:33.999364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:34.056328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:34.056927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.057208 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.057480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.058105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:34.060051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:34.083560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1192/2396] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:44:33 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:44:34.087674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:34.097806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a080d5-8492-4fff-afd5-f27fd1a22101 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:34.102357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:34.102973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.103251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.103521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.104155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:34.105738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:34.106954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:34.117171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1205/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:44:34.121428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:34.132234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4503d-8db6-460f-8dba-6ec9428762cf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:34.136970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:34.137761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.138038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.138309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.139005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:34.140579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eae2e-8377-4ac1-b485-5d56fb84c408 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:34.151072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1193/2398] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:44:34.155064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:34.164867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c986-d1c7-4390-b03c-68c0cd8d451f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:34.169373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:34.170011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.170286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.170556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] PATH_INFO: `/v3/projects/c75359bcb24b4beb8de99809d2bef6fa/users/75bdaeec19be45388613c7e22aa009db/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.185592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75359bcb24b4beb8de99809d2bef6fa, user_id=75bdaeec19be45388613c7e22aa009db, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:34.188298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:34.201568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1206/2399] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:44:34 2022] PUT /identity/v3/projects/c75359bcb24b4beb8de99809d2bef6fa/users/75bdaeec19be45388613c7e22aa009db/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:44:34.206479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730598f-7b72-4299-85f4-6c13fd7a79ba None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.206805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730598f-7b72-4299-85f4-6c13fd7a79ba None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.207083 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730598f-7b72-4299-85f4-6c13fd7a79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.226059 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b730598f-7b72-4299-85f4-6c13fd7a79ba None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:34.227363 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b730598f-7b72-4299-85f4-6c13fd7a79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:34.283944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1194/2400] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:34.294072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:34.295853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:34.353850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd9d44-94c7-4cc7-b1c9-917c5926b369 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:34.357703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:34.358326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:34.358602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:34.358903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:34.369073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:34.370621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:34.380358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1207/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:44:38.366266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.366865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.366959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.398957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] MFA Rules not processed for user `75bdaeec19be45388613c7e22aa009db`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:38.400275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:38.469735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1195/2402] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:38.474509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.484681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdab0-09fa-48bd-81ae-65f845c7f4ee service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.490319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.490970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.491247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.491518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.492149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.494086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.523050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1208/2403] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:44:38 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:44:38.527549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.540084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8049f2-ae0a-4f69-a4bd-d7b890839db8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.553503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:38.602850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.603398 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.603709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.603982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.604612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.606539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.630050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1196/2404] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:44:38 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:44:38.634146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.646160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bbe2-c6f1-44f4-a508-ba14918e3bd2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.653675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.654308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.654593 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.654903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.655554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.657221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.668357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1209/2405] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:44:38.672721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.683513 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54607325-8774-4ee2-b376-433c96eccfcf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.688163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.688813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.689088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.689358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] PATH_INFO: `/v3/projects/5ac9cfc464234f8aaf7391bd8462521b/users/0fbe37434f3c4df7a60387c767450505/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.704388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac9cfc464234f8aaf7391bd8462521b, user_id=0fbe37434f3c4df7a60387c767450505, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.707109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.719731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1197/2406] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:44:38 2022] PUT /identity/v3/projects/5ac9cfc464234f8aaf7391bd8462521b/users/0fbe37434f3c4df7a60387c767450505/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:44:38.729367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.740249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caa1a2-f66a-42eb-85f7-f78d182a43bd admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.796026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.796637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.796918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.797192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.797836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.799475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a6c45-40e3-48bd-be4a-b9a598927e5b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.810607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1210/2407] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:44:38.814817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:38.825542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a8f7e-b0a4-49fc-9731-952783ea9d37 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:38.830276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:38.830992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.831271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.831546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] PATH_INFO: `/v3/projects/5ac9cfc464234f8aaf7391bd8462521b/users/0fbe37434f3c4df7a60387c767450505/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.833516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac9cfc464234f8aaf7391bd8462521b, user_id=0fbe37434f3c4df7a60387c767450505, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:38.836210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:38.846558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1198/2408] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:44:38 2022] PUT /identity/v3/projects/5ac9cfc464234f8aaf7391bd8462521b/users/0fbe37434f3c4df7a60387c767450505/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:44:44.538080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6b379-e932-44b4-b53a-85f3160a3c56 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:44.538574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6b379-e932-44b4-b53a-85f3160a3c56 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:44.538787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6b379-e932-44b4-b53a-85f3160a3c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:44.575105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d1b6b379-e932-44b4-b53a-85f3160a3c56 None None] MFA Rules not processed for user `0fbe37434f3c4df7a60387c767450505`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:44.577155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d1b6b379-e932-44b4-b53a-85f3160a3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:44.612535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:44.683479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1211/2409] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:44.697992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:44.700794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:44.820080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f8d4-be8f-4527-b4d0-9ce5faf55a9b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:44.827107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:44.828040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:44.828601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:44.829093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:44.928838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:44.931610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f84ae2-6328-4e34-a7ad-d47813f660a8 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:44.949012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1199/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:47.120934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9394-e637-4d6d-ae26-14c0179e4005 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:47.121473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9394-e637-4d6d-ae26-14c0179e4005 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:47.121473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9394-e637-4d6d-ae26-14c0179e4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:47.143492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2fcd9394-e637-4d6d-ae26-14c0179e4005 None None] MFA Rules not processed for user `75bdaeec19be45388613c7e22aa009db`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:47.144996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2fcd9394-e637-4d6d-ae26-14c0179e4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:47.159116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1212/2411] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:47.458256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:47.458827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:47.458827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:47.468835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:47.486963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] MFA Rules not processed for user `072548785a38488db94ee05dfbf8c8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:47.491862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:47.585163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1200/2412] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:55.701279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:55.701822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:55.701939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:55.720349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] MFA Rules not processed for user `f594c9344b2c472ba33ceef87ec88a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:55.721579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:55.775566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1213/2413] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:44:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:57.573640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=23 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:04.743762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:04.814236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc12ef-e76f-46e0-9cc3-ed7c4bcb793a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:04.816385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:04.817828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:04.818254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:04.818993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:04.819436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:04.819777 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:04.821713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:04.829782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0b14e-2c8a-42c3-a8d4-ea3cb625347e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:04.833881 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:04.834482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:04.835019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:04.835368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:04.837036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:04.837155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:04.838630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:04.851867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1201/2414] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:04.852100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:04.853359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:04.866028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1214/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:45:14.846436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=26 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:14.860804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=51 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:20.567087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:20.575698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e03df-e9e9-4cf3-abb5-4ae322644387 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:20.579516 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:20.580191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:20.580539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:20.580877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:20.591045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:20.592558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:20.602556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1202/2416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:21.796275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:21.805592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593897a-f5dd-442d-9425-4ea4c1a42368 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:21.809272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:21.809871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:21.810153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:21.810426 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:21.812333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:21.883359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:21.884889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:21.894231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1215/2417] 10.222.0.42 () {60 vars in 1396 bytes} [Wed Aug 24 07:45:21 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:45:29.006404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:29.014908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf9ca-deb9-44db-a072-6094b72d35b8 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:29.019118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:29.019699 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:29.019970 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:29.020234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:29.021907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:29.082844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:29.084426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:29.095251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1203/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:31.890963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=28 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:32.154000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.155591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:32.170474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bbeb9-b153-4ebc-8071-2743400692ee service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.180533 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.181175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.181452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.181724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] PATH_INFO: `/v3/users/ea609a2504e64f6f9380061080976a6e` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.182922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea609a2504e64f6f9380061080976a6e)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.184918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.231374 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea609a2504e64f6f9380061080976a6e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.232020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea609a2504e64f6f9380061080976a6e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.232621 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea609a2504e64f6f9380061080976a6e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.244604 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea609a2504e64f6f9380061080976a6e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.252174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-278b0512-9358-464e-9628-75b208ad8b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea609a2504e64f6f9380061080976a6e'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.264154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1216/2419] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:45:32 2022] DELETE /identity/v3/users/ea609a2504e64f6f9380061080976a6e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:45:32.438049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.450836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2bdea-785f-444d-a66d-df4a5ced1776 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.508929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.509558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.509837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.510109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] PATH_INFO: `/v3/projects/d5672bdc355c4f63885138bd9c7f3497` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.511273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5672bdc355c4f63885138bd9c7f3497)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.513172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.558447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5672bdc355c4f63885138bd9c7f3497', 'request_id': 'req-9e02143f-edc5-42c5-ba30-0d2fb491442f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.565564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5672bdc355c4f63885138bd9c7f3497', 'request_id': 'req-9e02143f-edc5-42c5-ba30-0d2fb491442f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.571550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5672bdc355c4f63885138bd9c7f3497', 'request_id': 'req-9e02143f-edc5-42c5-ba30-0d2fb491442f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.582026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] The token cache is being invalidate because project d5672bdc355c4f63885138bd9c7f3497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:45:32.582312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.583022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9e02143f-edc5-42c5-ba30-0d2fb491442f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:45:32.584649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1204/2420] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:45:32 2022] DELETE /identity/v3/projects/d5672bdc355c4f63885138bd9c7f3497 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:45:32.589633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.589919 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.590195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.609539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:32.610777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:32.669089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1217/2421] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:45:32.677730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.678016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.678296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.699184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:32.700389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:32.708231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1205/2422] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:32.712575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.723128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b0f72-4bc3-40ff-9a46-68144ab1f8d2 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.727732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.728314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.728592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.728866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.729513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:45:32.729822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.731457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.746450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1218/2423] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:45:32.750757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.756444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:32.760758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7624a64-7576-4320-ba30-fdf502634ed0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.766652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.767604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.768176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.768577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.769617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.771683 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.798038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1206/2424] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:45:32 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:45:32.803220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.813267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcff7d-81db-4aa1-9bc1-e7c74ecc15a8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.868299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.868891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.869165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.869431 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.870052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.871982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.879608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:32.893724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1219/2425] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:45:32 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:45:32.897988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.909193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef039d1-52b0-4ab9-b2cb-035512a4c684 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.913682 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.914275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.914553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.914871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.915538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.917122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.928867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1207/2426] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:45:32.932759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.942596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc000abf-d8a6-460c-8c9e-73268ac43f51 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.947176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.947773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.948048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.948322 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.948963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.950543 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba07871-aa0b-4164-aace-ba1b8c49a18c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:32.960838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1220/2427] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:45:32.964796 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.975113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410640a7-d565-4754-8ae5-6092e4262d27 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:32.979653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:32.980313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.980586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.980854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] PATH_INFO: `/v3/projects/337840b04c0b4135a83063dfc5dbe5f1/users/a986ecc6e2644db8be8958b1eacfd5d6/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.995375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=337840b04c0b4135a83063dfc5dbe5f1, user_id=a986ecc6e2644db8be8958b1eacfd5d6, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:32.998004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:33.010342 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1208/2428] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:45:32 2022] PUT /identity/v3/projects/337840b04c0b4135a83063dfc5dbe5f1/users/a986ecc6e2644db8be8958b1eacfd5d6/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:45:33.015317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7e0d-ddf7-4482-82d3-3d6fe9f997f9 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:33.015608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7e0d-ddf7-4482-82d3-3d6fe9f997f9 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:33.015890 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7e0d-ddf7-4482-82d3-3d6fe9f997f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:33.034771 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6e5d7e0d-ddf7-4482-82d3-3d6fe9f997f9 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:33.035973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6e5d7e0d-ddf7-4482-82d3-3d6fe9f997f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:33.094844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1221/2429] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:33.105049 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:33.106756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:33.162652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d317376-cff7-4c22-82a3-05ab9e424b7a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:33.166362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:33.167077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:33.167355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:33.167629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:33.176614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:33.178060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:33.187462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1209/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:37.014740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c29eb7-b55e-489e-b423-2a90f4aef272 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:37.015244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c29eb7-b55e-489e-b423-2a90f4aef272 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:37.015376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c29eb7-b55e-489e-b423-2a90f4aef272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:37.044592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a9c29eb7-b55e-489e-b423-2a90f4aef272 None None] MFA Rules not processed for user `a986ecc6e2644db8be8958b1eacfd5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:37.045888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a9c29eb7-b55e-489e-b423-2a90f4aef272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:37.117904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1222/2431] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:37.132761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:37.134274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:37.209823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36af86-34db-422b-8a2e-3dee10a232c4 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:37.215455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:37.216060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:37.216334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:37.216598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:37.224505 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:37.226084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:37.235399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1210/2432] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:38.581785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd675-da09-418b-9860-104653a6b89c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:38.582306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd675-da09-418b-9860-104653a6b89c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:38.582306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd675-da09-418b-9860-104653a6b89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:38.602097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-664bd675-da09-418b-9860-104653a6b89c None None] MFA Rules not processed for user `a986ecc6e2644db8be8958b1eacfd5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:38.603334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-664bd675-da09-418b-9860-104653a6b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:38.610350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1223/2433] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:45:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:38.919235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:38.920632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:38.932665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af89ba-36f4-46b0-9f59-299d5c38f65e service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:38.936196 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:38.936782 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:38.937062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:38.937309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:38.939533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:38.949302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:38.950613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:38.959582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1211/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:39.280041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd675-da09-418b-9860-104653a6b89c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:39.281448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-664bd675-da09-418b-9860-104653a6b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:39.388925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:39.395382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd675-da09-418b-9860-104653a6b89c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:39.402163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:39.403014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:39.403487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:39.403820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:39.405750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:39.418770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:39.420731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:39.432780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1224/2435] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:40.864598 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:40.882062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0da5e-8bc2-4b67-bb29-53b53c693117 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:40.888497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:40.889209 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:40.889564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:40.889933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:40.891987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:40.915664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:40.961369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:40.963151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:40.975122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1212/2436] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:41.222529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:41.233277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bdd74-3460-49f3-9650-2d6d2d3d29e6 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:41.237811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:41.238424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:41.239056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:41.239488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:41.241652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:41.255233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:41.256828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:41.269670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1225/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:42.138968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:42.151962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045efd-b7b6-4867-bcc8-6fd018529e02 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:42.159351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:42.160000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:42.160292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:42.160569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:42.162362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:42.175994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:42.177607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf2969-4d51-4b48-aa83-2c47262ae313 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:42.190864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1213/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:45:47.764176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:47.765660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:47.783478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b387-c60a-42d7-98c7-3fcad67a70c5 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:47.789080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:47.789694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:47.789961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:47.790233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:47.792059 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:47.805826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:47.807500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485b161-2abb-4323-b96e-d44125355b5d service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:47.818458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1226/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:52.185559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=24 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:57.814598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=16 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:08.581944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:08.583235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:08.584496 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:08.607716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] MFA Rules not processed for user `760115e8fb554582a016445f9aadc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:08.609383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:08.675478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1214/2440] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:13.197372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:13.197909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:13.197909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:13.219218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] MFA Rules not processed for user `f594c9344b2c472ba33ceef87ec88a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:13.220370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:13.294926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1227/2441] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:14.866425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:14.882961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afac38-00bf-4b69-a163-749e471f3128 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:14.889948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:14.891189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:14.891693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:14.892391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:14.894541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:14.926838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:14.940192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:14.952599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1215/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:21.158401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:21.169037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341bcf7-85fe-4464-8b62-e51271a6be6d None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:21.172569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:21.173425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:21.173702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:21.173973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:21.175714 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:21.187229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:21.189154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:21.198603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1228/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:46:24.700405 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:24.712604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200ff9-64f8-4757-9546-3e81c2456913 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:24.717718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:24.718430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:24.718750 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:24.719031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:24.721544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:24.733371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:24.735024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:24.744851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1216/2444] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:24.771845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:24.774170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:24.796221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903a57-fa1b-487d-acda-741a0884e22d service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:24.802232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:24.802917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:24.803238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:24.803509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] PATH_INFO: `/v3/users/072548785a38488db94ee05dfbf8c8ec` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:24.804625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] RBAC: Authorizing `identity:delete_user(user_id=072548785a38488db94ee05dfbf8c8ec)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:24.806549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:24.853019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072548785a38488db94ee05dfbf8c8ec'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:24.853686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072548785a38488db94ee05dfbf8c8ec'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:24.854296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072548785a38488db94ee05dfbf8c8ec'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:24.865849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072548785a38488db94ee05dfbf8c8ec'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:24.873174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-98a5902e-4e89-447a-b3e8-b628de4091de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072548785a38488db94ee05dfbf8c8ec'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:24.886024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1229/2445] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:46:24 2022] DELETE /identity/v3/users/072548785a38488db94ee05dfbf8c8ec => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:25.063917 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.076651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c288c6a-2b1c-4bf3-ad44-8e96eac4df39 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.137490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.138120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.138399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.138723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] PATH_INFO: `/v3/projects/3e9b29dbfd404c008fe871808d41e059` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.139962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9b29dbfd404c008fe871808d41e059)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.142449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.189887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9b29dbfd404c008fe871808d41e059', 'request_id': 'req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:25.198633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9b29dbfd404c008fe871808d41e059', 'request_id': 'req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:25.204748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9b29dbfd404c008fe871808d41e059', 'request_id': 'req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:25.214679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] The token cache is being invalidate because project 3e9b29dbfd404c008fe871808d41e059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:46:25.214988 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:25.215606 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6ba0c16e-a8fe-4c85-b1dc-e48e622eea75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:25.217051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1217/2446] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:46:25 2022] DELETE /identity/v3/projects/3e9b29dbfd404c008fe871808d41e059 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:25.221937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.222215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.222484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.241918 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:25.243139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:25.301208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1230/2447] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:25.308938 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.309228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.309508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.329451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:25.330649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:25.338483 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1218/2448] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:25.342542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.349256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:25.353198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cae73-0703-4a38-8b60-c4ad8d6ea132 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.359412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.360773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.361218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.361599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.362525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:46:25.362857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.364473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.383386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1231/2449] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:46:25.383527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.388926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:25.396940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a562f4c-2be6-41a7-8065-96c56880aaa2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.404042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.404842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.405216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.405584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.406343 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.408592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.444933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1219/2450] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:46:25.446071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.459169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afb41e-3c6a-4ff9-ad47-64cb910bcfa0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.518509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.519152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.519454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.519731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.520357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.522280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.543584 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1232/2451] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:46:25.548670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.559353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58ae2-d3c4-49e0-b411-d3331b487f60 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.563932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.564535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.564845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.565120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.565761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.567394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.577847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1220/2452] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:25.582469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.593119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8665-1831-4680-91ab-6643e15e3d71 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.598430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.599076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.599346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.599619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.600257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.601853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96178f2a-456f-4e08-852f-65409de7966d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.613549 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1233/2453] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:25.617814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.628146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38e51a-be86-4972-9def-2df5c497876e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.633022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.633698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.633979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.634256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] PATH_INFO: `/v3/projects/18816cda9520496da8ff1df6b6b303f6/users/d86166e5d73142809827c3b0ec5fb8eb/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.650244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=18816cda9520496da8ff1df6b6b303f6, user_id=d86166e5d73142809827c3b0ec5fb8eb, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.652921 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.669686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1221/2454] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:25 2022] PUT /identity/v3/projects/18816cda9520496da8ff1df6b6b303f6/users/d86166e5d73142809827c3b0ec5fb8eb/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:46:25.681270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.682140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.682493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.710412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] MFA Rules not processed for user `d86166e5d73142809827c3b0ec5fb8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:25.711729 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:25.780459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1234/2455] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:25.785623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.846597 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88d2be-8e22-4416-978c-6b3a4d928fbd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.851169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.851762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.852036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.852301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.852913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.854820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.880649 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1222/2456] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:46:25.885313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.899942 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250bd8-119a-4463-a181-7011bc01c9bd None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:25.956359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:25.956880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:25.957159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:25.957433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:25.958050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:25.960003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:25.967653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:25.982253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1235/2457] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:46:25 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:46:25.986325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:25.996602 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728227a-cc8b-430c-8b59-93e5e5018358 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:26.001152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:26.001742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.002020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.002292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.002962 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:26.004556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:26.015226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1223/2458] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:26.019866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:26.029859 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbde73b-54de-4605-8676-c5fee8d37608 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:26.035127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:26.035794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.036078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.036356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] PATH_INFO: `/v3/projects/705c386e58f8428cb9b0b1790df988ea/users/360f76de020c47fa835964192ce9316e/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.051321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=705c386e58f8428cb9b0b1790df988ea, user_id=360f76de020c47fa835964192ce9316e, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:26.053988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:26.066989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1236/2459] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:26 2022] PUT /identity/v3/projects/705c386e58f8428cb9b0b1790df988ea/users/360f76de020c47fa835964192ce9316e/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:26.071998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:26.083124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161030ef-3127-4b1a-9955-edb4f803f163 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:26.141542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:26.142151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.142429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.142719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.143366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:26.144964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e89f3-ca5d-4b5d-bcc7-a27acb879679 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:26.154767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1224/2460] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:26.160075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:26.171663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc776af-2bf6-47b3-8a53-99812b80da38 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:26.177037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:26.177904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.178289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.179161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] PATH_INFO: `/v3/projects/705c386e58f8428cb9b0b1790df988ea/users/360f76de020c47fa835964192ce9316e/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.181084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=705c386e58f8428cb9b0b1790df988ea, user_id=360f76de020c47fa835964192ce9316e, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:26.184160 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:26.196715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1237/2461] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:26 2022] PUT /identity/v3/projects/705c386e58f8428cb9b0b1790df988ea/users/360f76de020c47fa835964192ce9316e/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:26.209146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.209365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.209639 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.243882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] MFA Rules not processed for user `360f76de020c47fa835964192ce9316e`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:26.245128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:26.249680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:26.329499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1225/2462] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:26.340542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:26.342271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:26.402135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230abfb-ac7d-498a-98bf-72b972160f1d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:26.405822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:26.406446 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:26.406781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:26.407053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:26.465090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:26.466455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:26.476105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1238/2463] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:28.042029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:28.044893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:28.110054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003927b-864e-4a8d-8aac-b22a27039c57 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:28.114722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:28.115353 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:28.115634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:28.116081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:28.118462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:28.185442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:28.187723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec11084-6429-4a24-aa1c-b48bb742ebc9 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:28.198588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1226/2464] 10.222.0.42 () {60 vars in 1396 bytes} [Wed Aug 24 07:46:28 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:31.177974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:31.180242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:31.202513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04837f4-d7ea-4cc2-94c0-69e27032161f service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:31.206973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:31.207586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:31.207866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:31.208139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:31.217627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:31.219767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:31.229480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1239/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:38.193672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=75 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:41.225038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=5,keystone:SELECT=71 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:51.807320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:51.807821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:51.807914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:51.827360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:51.828579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:51.883322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1227/2466] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:46:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:51.928493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:51.929873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:52.003222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815e486-0419-4e9f-9eb6-1518033eb573 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:52.008334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:52.008949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:52.009230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:52.009504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:52.018900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:52.020520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a7694-1b9f-4b30-aae5-a4410b1bcff7 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:52.029312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1240/2467] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:46:56.291106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:56.370317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc052dc-d263-4e8c-aa91-af75efa6e9c3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:56.377959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:56.378612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:56.378943 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:56.379221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] PATH_INFO: `/v3/users/d86166e5d73142809827c3b0ec5fb8eb` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:56.380396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86166e5d73142809827c3b0ec5fb8eb)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:56.382375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:56.445754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86166e5d73142809827c3b0ec5fb8eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:56.446525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86166e5d73142809827c3b0ec5fb8eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:56.455080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86166e5d73142809827c3b0ec5fb8eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:56.456100 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86166e5d73142809827c3b0ec5fb8eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:56.472233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86166e5d73142809827c3b0ec5fb8eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:56.490226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1228/2468] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:46:56 2022] DELETE /identity/v3/users/d86166e5d73142809827c3b0ec5fb8eb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:56.495886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:56.496822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:56.497106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:56.519619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:56.520745 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:56.598836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1241/2469] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:56.608288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:56.609738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:56.647198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:56.685423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600eced-dad1-4b6f-9725-c733c3fe5f54 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:56.690027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:56.690648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:56.690965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:56.691242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:56.701949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:56.703484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:56.715032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1229/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:57.070725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:57.072313 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:57.101774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0431c80-410e-4f24-9bfe-22d80adfda41 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:57.107692 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:57.108519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:57.109524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:57.109905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] PATH_INFO: `/v3/projects/18816cda9520496da8ff1df6b6b303f6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:57.112422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] RBAC: Authorizing `identity:delete_project(project_id=18816cda9520496da8ff1df6b6b303f6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:57.115240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:57.199073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18816cda9520496da8ff1df6b6b303f6', 'request_id': 'req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.226894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18816cda9520496da8ff1df6b6b303f6', 'request_id': 'req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.226894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18816cda9520496da8ff1df6b6b303f6', 'request_id': 'req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.234904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] The token cache is being invalidate because project 18816cda9520496da8ff1df6b6b303f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:46:57.235396 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.241641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.244355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1242/2471] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:46:57 2022] DELETE /identity/v3/projects/18816cda9520496da8ff1df6b6b303f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:57.248720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:57.269322 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284856fa-487f-4854-aa96-5f17fd344a8d service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:57.337578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:57.337578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:57.337578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:57.337578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] PATH_INFO: `/v3/users/360f76de020c47fa835964192ce9316e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:57.338430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360f76de020c47fa835964192ce9316e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:57.339841 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:57.398055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360f76de020c47fa835964192ce9316e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.399073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360f76de020c47fa835964192ce9316e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.406830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360f76de020c47fa835964192ce9316e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.408215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360f76de020c47fa835964192ce9316e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.421532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bb201cb2-d2fc-43cf-ab0c-e4f672456161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360f76de020c47fa835964192ce9316e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.437081 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1230/2472] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:46:57 2022] DELETE /identity/v3/users/360f76de020c47fa835964192ce9316e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:57.765541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:57.767658 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:57.782458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d22050-a037-4ad2-8d02-0a2c5ae1273b admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:57.802483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:57.860110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:57.860870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:57.861149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:57.861418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] PATH_INFO: `/v3/projects/705c386e58f8428cb9b0b1790df988ea` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:57.862560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705c386e58f8428cb9b0b1790df988ea)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:57.864505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:57.931297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705c386e58f8428cb9b0b1790df988ea', 'request_id': 'req-2cd66977-b953-4d47-ba1f-a3272bad7e90'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.941092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705c386e58f8428cb9b0b1790df988ea', 'request_id': 'req-2cd66977-b953-4d47-ba1f-a3272bad7e90'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.949143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705c386e58f8428cb9b0b1790df988ea', 'request_id': 'req-2cd66977-b953-4d47-ba1f-a3272bad7e90'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.961724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] The token cache is being invalidate because project 705c386e58f8428cb9b0b1790df988ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:46:57.962128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.963030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-2cd66977-b953-4d47-ba1f-a3272bad7e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:46:57.964861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1243/2473] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:46:57 2022] DELETE /identity/v3/projects/705c386e58f8428cb9b0b1790df988ea => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:57.970433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:57.970935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:57.971295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:57.995572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:57.996863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:58.058891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:58.094127 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1231/2474] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:58.095642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.095933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.096213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.129526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:58.130705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:58.141694 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1244/2475] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:58.146119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.157171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaf64e-83c8-4529-a23d-268aeaadc535 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.161882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.164421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.164421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.164421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.164421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:46:58.164421 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.166038 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.183404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1232/2476] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:46:58.187885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.198646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aba9e-b087-4561-8a42-85a66f089988 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.203598 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.204289 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.204565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.204838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.205540 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.207473 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.237785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1245/2477] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:46:58.241608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.253020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58773c-582d-4504-b473-811490b24055 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.322906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.323858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.324428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.324920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.325701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.328153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.357619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1233/2478] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:46:58.361970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.378545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642088-06dc-4048-8d33-1b98f44069e8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.385222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.386269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.386551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.386856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.387518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.389136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.399932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1246/2479] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:58.403989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.415563 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb380964-9375-43c7-84e9-34ea46ca4abb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.420249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.421010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.421295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.421569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.422215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.424766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917f28f-5733-4201-8761-b30b158c5e80 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.441674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1234/2480] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:58.446136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.457204 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b8ae-f568-4cd6-8baa-a64f2970a281 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.463093 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.464068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.464454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.464800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] PATH_INFO: `/v3/projects/6c2ada97052b4b0ab3a16f7f86a620b6/users/db3ed914112d44fd8ac34d26bb511640/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.482755 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2ada97052b4b0ab3a16f7f86a620b6, user_id=db3ed914112d44fd8ac34d26bb511640, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.484743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.510069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1247/2481] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:58 2022] PUT /identity/v3/projects/6c2ada97052b4b0ab3a16f7f86a620b6/users/db3ed914112d44fd8ac34d26bb511640/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:46:58.512234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.512629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.512984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.550524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] MFA Rules not processed for user `db3ed914112d44fd8ac34d26bb511640`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:58.551916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:58.629034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1235/2482] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:46:58.636345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.685603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:58.716351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c840029-5c5f-4979-bfd3-e5f593acef8c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.722444 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.723149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.723435 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.723709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.724350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.726741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.758032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1248/2483] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/projects => generated 401 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:46:58.762387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.773834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95364df-9d4b-4836-9dee-e97e5442f09a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.823774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:58.845275 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.846000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.846386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.846808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.847778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.849815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.876315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1236/2484] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:46:58 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:46:58.877794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.888451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f658-e536-424e-b5b1-b8173cf88eb7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.899601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.900319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.900672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.901011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.901663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.903680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.921872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.927113 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1249/2485] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:58.933460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627725ff-aeae-4b13-8164-d3a40a5683ed admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:58.938850 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:58.939669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:58.939956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:58.940235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] PATH_INFO: `/v3/projects/a9720f6145f34fa5b44435b49097da1e/users/5937179c296648bb8c793a3deb54a9da/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:58.958595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9720f6145f34fa5b44435b49097da1e, user_id=5937179c296648bb8c793a3deb54a9da, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:58.961433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:58.981320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1237/2486] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:58 2022] PUT /identity/v3/projects/a9720f6145f34fa5b44435b49097da1e/users/5937179c296648bb8c793a3deb54a9da/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:46:58.982998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:58.995798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec407f-3aeb-44a1-ba79-4043e73c138c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:59.071922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:59.072535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:59.072812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:59.073079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:59.073721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:59.075344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5cbcd-1225-4447-b73a-7671cbc0b2cc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:59.090734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1250/2487] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:59.092878 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:59.106540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170e07a-ef83-44e8-9741-710dd2fd59bb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:59.111471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:59.112283 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:59.112566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:59.112837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] PATH_INFO: `/v3/projects/a9720f6145f34fa5b44435b49097da1e/users/5937179c296648bb8c793a3deb54a9da/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:59.114946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9720f6145f34fa5b44435b49097da1e, user_id=5937179c296648bb8c793a3deb54a9da, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:59.123898 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:59.136677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1238/2488] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:46:59 2022] PUT /identity/v3/projects/a9720f6145f34fa5b44435b49097da1e/users/5937179c296648bb8c793a3deb54a9da/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:46:59.147388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:59.147702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:59.147984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:59.193864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] MFA Rules not processed for user `5937179c296648bb8c793a3deb54a9da`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:59.197198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:59.311128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1251/2489] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:46:59 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:59.319686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:59.321572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:59.412628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1d3c-e55a-438e-b2b7-8667be9b4e47 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:59.416477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:59.417213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:59.417569 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:59.417913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:59.432859 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:59.434424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:59.447200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1239/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:59 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:59.537715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:59.548810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33321e89-57de-4336-86c1-8e91500e10b7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:59.552837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:59.553466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:59.553817 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:59.554141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:59.596315 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:59.630400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:59.632022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d69c16-fd0b-45d4-a1f4-26677a3558f5 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:59.642215 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1252/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:03.758423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:03.821629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf589e15-5a6f-4f41-ba12-6a09d3762f37 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:03.826599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:03.827245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:03.827515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:03.827783 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] PATH_INFO: `/v3/users/db3ed914112d44fd8ac34d26bb511640` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:03.828889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3ed914112d44fd8ac34d26bb511640)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:03.831542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:03.870397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:03.877837 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3ed914112d44fd8ac34d26bb511640'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:03.878763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3ed914112d44fd8ac34d26bb511640'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:03.885736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3ed914112d44fd8ac34d26bb511640'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:03.886365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3ed914112d44fd8ac34d26bb511640'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:03.898471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3ed914112d44fd8ac34d26bb511640'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:03.910770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1240/2492] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:03 2022] DELETE /identity/v3/users/db3ed914112d44fd8ac34d26bb511640 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:03.915387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:03.915845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:03.915961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:03.935123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:03.936310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:03.992418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1253/2493] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:04.002631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:04.004072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:04.066913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee16f-71df-4c0d-a4a9-68d310b7eac3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:04.070651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:04.071296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.071578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.072004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:04.086950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:04.089023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:04.099685 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1241/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:04.372401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:04.374099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:04.392696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b8e73-54c8-460a-a37d-7dd6a998d160 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:04.398077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:04.398810 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.399100 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.399375 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] PATH_INFO: `/v3/projects/6c2ada97052b4b0ab3a16f7f86a620b6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:04.400504 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2ada97052b4b0ab3a16f7f86a620b6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:04.402430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:04.457373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2ada97052b4b0ab3a16f7f86a620b6', 'request_id': 'req-7823206f-760d-4688-bee3-8decba398865'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.465344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2ada97052b4b0ab3a16f7f86a620b6', 'request_id': 'req-7823206f-760d-4688-bee3-8decba398865'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.472603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2ada97052b4b0ab3a16f7f86a620b6', 'request_id': 'req-7823206f-760d-4688-bee3-8decba398865'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.484923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] The token cache is being invalidate because project 6c2ada97052b4b0ab3a16f7f86a620b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:04.485285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.485987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.487559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1254/2495] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:04 2022] DELETE /identity/v3/projects/6c2ada97052b4b0ab3a16f7f86a620b6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:04.491857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:04.502913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5712261-c970-4cd4-8db1-ff367259efc7 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:04.564937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:04.565564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.565843 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.566117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] PATH_INFO: `/v3/users/5937179c296648bb8c793a3deb54a9da` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:04.567287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5937179c296648bb8c793a3deb54a9da)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:04.569230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:04.620518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5937179c296648bb8c793a3deb54a9da'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.621242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5937179c296648bb8c793a3deb54a9da'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.629311 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5937179c296648bb8c793a3deb54a9da'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.630034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5937179c296648bb8c793a3deb54a9da'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.634901 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:04.645251 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5937179c296648bb8c793a3deb54a9da'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:04.658622 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1242/2496] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:04 2022] DELETE /identity/v3/users/5937179c296648bb8c793a3deb54a9da => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:04.706320 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:04.707831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:04.765793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823206f-760d-4688-bee3-8decba398865 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:04.775886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:04.820413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:04.821271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.821552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.821829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:04.823656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:04.834839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:04.836775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd2203-cd1a-411c-8eef-082f49985af6 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:04.845468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1255/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:04.956914 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:04.958895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:04.978850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1402cf-75ad-4ba8-acf8-6452a6c5ff39 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.979370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1402cf-75ad-4ba8-acf8-6452a6c5ff39 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.979489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1402cf-75ad-4ba8-acf8-6452a6c5ff39 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:04.980752 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1256/2498] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:47:04.987429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:04.987805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:04.987980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.017203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.018437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.056183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191ec7-c65d-4e8d-b123-7cdcefffdfd6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.061510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.062230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.062584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.062963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] PATH_INFO: `/v3/projects/a9720f6145f34fa5b44435b49097da1e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.064729 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9720f6145f34fa5b44435b49097da1e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.066845 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.099937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1257/2499] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:47:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.114400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.116163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.128825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9720f6145f34fa5b44435b49097da1e', 'request_id': 'req-f6f4ae4f-921d-4de3-900c-609873dde632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:05.143132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9720f6145f34fa5b44435b49097da1e', 'request_id': 'req-f6f4ae4f-921d-4de3-900c-609873dde632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:05.155714 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9720f6145f34fa5b44435b49097da1e', 'request_id': 'req-f6f4ae4f-921d-4de3-900c-609873dde632'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:05.167853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] The token cache is being invalidate because project a9720f6145f34fa5b44435b49097da1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:05.168266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:05.168924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6f4ae4f-921d-4de3-900c-609873dde632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:05.170445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1243/2500] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:04 2022] DELETE /identity/v3/projects/a9720f6145f34fa5b44435b49097da1e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:05.175403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf747c-97a7-4059-81fe-a1df131236b3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.175686 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf747c-97a7-4059-81fe-a1df131236b3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.175968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf747c-97a7-4059-81fe-a1df131236b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.186612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97174ca8-e5e5-4c4d-8e25-e73e781408d7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.195216 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1dbf747c-97a7-4059-81fe-a1df131236b3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.196518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1dbf747c-97a7-4059-81fe-a1df131236b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.236857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.237388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.237657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.237935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.239750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.256447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1244/2501] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.257869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:05.265298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.265588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.265867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.286211 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.287468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.296103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1245/2502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.300424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.304669 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:05.310749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dd314-5089-4ed8-bd9a-4fd67b06cd17 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.312560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.314586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.317160 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.317752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.318032 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.318308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.319008 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:05.319725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.321539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.326068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1258/2503] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:05.337333 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1246/2504] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:47:05.338433 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.349816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e1a3-d46b-4cb5-af7a-4414e0c04548 service glance] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.354490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.355131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.355412 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.355684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.356311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.358229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ebc3c-a6cd-42d5-a967-bf4cd53ed86e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.387130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1259/2505] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:05.391073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.398323 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069c53-3f46-415d-8aaa-90f16c32b055 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.398712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069c53-3f46-415d-8aaa-90f16c32b055 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.399071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069c53-3f46-415d-8aaa-90f16c32b055 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.400523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1260/2506] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:05.402248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef02c6-a955-4299-93d1-b62a14ec9fcc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.404986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.405364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.405710 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.425977 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.427226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.461960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.462576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.462884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.463159 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.463791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.465708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.479448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1261/2507] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.490801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1247/2508] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:05.493233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.493428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.494981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.503697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3972d44-391a-4f9e-86af-8a8f2db7c3e0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.505934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155cb1-8802-4c60-b422-147becb6a293 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.508515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.509272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.509562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.509833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.510040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.510502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.510668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.510982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.511256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.512139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.519196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.520627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.523932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1248/2509] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:05.527934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.529170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1262/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:05.537736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1aace-0e6b-42e4-a4a1-d0f63375029e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.542468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.543096 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.543380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.543655 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.544296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.545888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294177d-423e-409f-9135-e4307767e210 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.556155 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1249/2511] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:47:05.560339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.572731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543959-6b18-4e8c-9ed3-195b6f0add28 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.577479 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.578138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.583636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.583636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] PATH_INFO: `/v3/projects/1bbc805ab5f042d697040c5eead4e48f/users/8925efeabeb84b5baab41f169920f8ee/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.596845 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbc805ab5f042d697040c5eead4e48f, user_id=8925efeabeb84b5baab41f169920f8ee, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.600643 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad698f3-ebee-48a0-9d1f-1de17457339c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.614288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1263/2512] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:05 2022] PUT /identity/v3/projects/1bbc805ab5f042d697040c5eead4e48f/users/8925efeabeb84b5baab41f169920f8ee/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:05.614493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26004-66f6-4c6e-84b8-8e68714964a0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.614828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26004-66f6-4c6e-84b8-8e68714964a0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.615126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26004-66f6-4c6e-84b8-8e68714964a0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.616472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1250/2513] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:47:05.621832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.622117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.622401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.624536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.624829 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.625112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.642931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.644206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.653713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] MFA Rules not processed for user `8925efeabeb84b5baab41f169920f8ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:05.654935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:05.693210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:05.713058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1264/2514] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.723154 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1251/2515] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:05.723537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.727413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.774952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:05.778811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64accd-c4c6-4a95-80f0-25b3d34d71d4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.782464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.783116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.783395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.783664 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.790137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e9567-8780-46b8-b687-cf09f8d4cf4a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.793974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.794919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.795448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.795584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.795792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.796068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.796691 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.799410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.807181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1265/2516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:05.828029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1252/2517] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:47:05.832141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.843293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474d0ff-b262-4914-9304-541d03cb3b62 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.901032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.901637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.901916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.902193 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.902842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.904874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.927047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1266/2518] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:05 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:05.931325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.946526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943ea54-2a86-4be9-97db-5027edd7ba10 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.951264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.951870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.952156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.952429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:05.953072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:05.954676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:05.965123 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1253/2519] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:05.969285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:05.979648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27c3a4-96e5-40d5-8f3b-b2df8bfd50c4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:05.984297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:05.984948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:05.985751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:05.986064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] PATH_INFO: `/v3/projects/4b3e30ae10b940bd9de87879234cf519/users/005664b5dd44417da00070949f701f04/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.002163 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3e30ae10b940bd9de87879234cf519, user_id=005664b5dd44417da00070949f701f04, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:06.004913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:06.017866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1267/2520] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:05 2022] PUT /identity/v3/projects/4b3e30ae10b940bd9de87879234cf519/users/005664b5dd44417da00070949f701f04/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:06.022046 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:06.033467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cda72a-8db3-4849-badb-221624af9f5d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:06.092834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:06.093522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:06.093872 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:06.094215 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.095512 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:06.097177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626b122-8808-4af0-812c-d84653b8bfac admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:06.107236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1254/2521] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:06.112146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:06.122886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47debda9-fee2-4954-8ee6-c2140ba7cc7f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:06.127759 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:06.128483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:06.128831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:06.129178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] PATH_INFO: `/v3/projects/4b3e30ae10b940bd9de87879234cf519/users/005664b5dd44417da00070949f701f04/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.131246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3e30ae10b940bd9de87879234cf519, user_id=005664b5dd44417da00070949f701f04, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:06.133914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:06.145948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1268/2522] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:06 2022] PUT /identity/v3/projects/4b3e30ae10b940bd9de87879234cf519/users/005664b5dd44417da00070949f701f04/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:06.155234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:06.155527 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:06.155807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.188212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] MFA Rules not processed for user `005664b5dd44417da00070949f701f04`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:06.189665 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:06.282737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1255/2523] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:06.299403 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:06.301279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:06.361577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:06.375736 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa9d7e-1238-4bf9-a56f-c4f2513b1e6c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:06.379391 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:06.379992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:06.380270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:06.380542 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.389355 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:06.391029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33d8d-89c1-4608-9126-44c575c36704 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:06.401663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1269/2524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:06.486143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:06.495694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e231d3-c064-4d2d-a221-05a2379ca0c7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:06.499185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:06.499779 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:06.500057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:06.500328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:06.540995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:06.558254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:06.559738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:06.569232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1256/2525] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:08.260295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:08.260782 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:08.260883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:08.279813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:08.281013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:08.335603 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1270/2526] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:47:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:08.348339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:08.349904 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:08.419174 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b6a9b-4002-48d0-b0b2-b7188da98cd1 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:08.424954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:08.425558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:08.425844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:08.426119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:08.434055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:08.435673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7caa6-c424-4205-924a-f6f6548dc5c1 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:08.443554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1257/2527] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:47:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:47:11.476489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:11.543088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6fc55-344a-4724-a9cc-c94e51277b3c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:11.548599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:11.549210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:11.549489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:11.549809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] PATH_INFO: `/v3/users/8925efeabeb84b5baab41f169920f8ee` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:11.550998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8925efeabeb84b5baab41f169920f8ee)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:11.552936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:11.612690 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8925efeabeb84b5baab41f169920f8ee'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:11.613372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8925efeabeb84b5baab41f169920f8ee'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:11.613991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8925efeabeb84b5baab41f169920f8ee'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:11.630663 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8925efeabeb84b5baab41f169920f8ee'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:11.640687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8925efeabeb84b5baab41f169920f8ee'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:11.659132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1271/2528] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:11 2022] DELETE /identity/v3/users/8925efeabeb84b5baab41f169920f8ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:11.663724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:11.664681 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:11.665120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:11.685106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:11.686630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:11.751236 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1258/2529] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:11.761367 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:11.762963 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:11.779680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:11.837989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41f973-668e-4325-86b2-3c383067d18e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:11.841814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:11.842439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:11.842742 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:11.843022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:11.853652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:11.855155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:11.866268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1272/2530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:12.181401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:12.183124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:12.202574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b5382-7d0c-45cf-b60e-dd07ccdf7fa2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:12.209834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:12.211767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:12.212964 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:12.213984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] PATH_INFO: `/v3/projects/1bbc805ab5f042d697040c5eead4e48f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:12.215510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbc805ab5f042d697040c5eead4e48f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:12.217487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:12.272716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbc805ab5f042d697040c5eead4e48f', 'request_id': 'req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.280517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbc805ab5f042d697040c5eead4e48f', 'request_id': 'req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.288255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbc805ab5f042d697040c5eead4e48f', 'request_id': 'req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.293718 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:12.303756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] The token cache is being invalidate because project 1bbc805ab5f042d697040c5eead4e48f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:12.304539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.305886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.307689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1259/2531] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:12 2022] DELETE /identity/v3/projects/1bbc805ab5f042d697040c5eead4e48f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:12.311703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:12.323938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85485435-6aaf-4ba5-bb9a-1149ac84e957 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:12.390227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:12.390994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:12.391304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:12.391610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] PATH_INFO: `/v3/users/005664b5dd44417da00070949f701f04` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:12.392800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=005664b5dd44417da00070949f701f04)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:12.396027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:12.450129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005664b5dd44417da00070949f701f04'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.451088 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005664b5dd44417da00070949f701f04'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.451698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005664b5dd44417da00070949f701f04'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.467317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005664b5dd44417da00070949f701f04'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.477161 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-a917d055-1b90-479f-8732-9cb14838ec97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005664b5dd44417da00070949f701f04'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.502187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1273/2532] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:12 2022] DELETE /identity/v3/users/005664b5dd44417da00070949f701f04 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:12.825780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:12.828133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:12.847482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc49bf-9c66-49b2-b6d6-e1642f39cd63 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:12.910798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:12.911441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:12.911716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:12.911989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] PATH_INFO: `/v3/projects/4b3e30ae10b940bd9de87879234cf519` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:12.913142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3e30ae10b940bd9de87879234cf519)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:12.915040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:12.964858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3e30ae10b940bd9de87879234cf519', 'request_id': 'req-8aa808e2-51a1-4e4b-8458-88010e47fc22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.976307 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3e30ae10b940bd9de87879234cf519', 'request_id': 'req-8aa808e2-51a1-4e4b-8458-88010e47fc22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.983741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3e30ae10b940bd9de87879234cf519', 'request_id': 'req-8aa808e2-51a1-4e4b-8458-88010e47fc22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.995670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] The token cache is being invalidate because project 4b3e30ae10b940bd9de87879234cf519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:12.996047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.996761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8aa808e2-51a1-4e4b-8458-88010e47fc22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:12.998631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1260/2533] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:12 2022] DELETE /identity/v3/projects/4b3e30ae10b940bd9de87879234cf519 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:13.004177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.004535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.004880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.031145 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:13.032434 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:13.088891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:13.135013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1274/2534] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:13.142518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.142839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.143103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.163635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:13.164856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:13.173723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1261/2535] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:13.177999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.189087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decc566-2c00-41d3-989f-3193c840e119 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.193857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.194591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.194969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.195241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.195912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:13.196199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.197825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.213640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1275/2536] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:47:13.215235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.224880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a1087-c4ee-4d3a-afe9-59e96fdb8956 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.229761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.230507 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.230823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.231097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.231741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.233662 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.263556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1262/2537] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:13.267842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.278774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d887f-7bb1-4fb1-af18-ad1451574ed4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.339127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.339656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.339932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.340206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.340839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.343013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.363991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1276/2538] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:13.368754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.381298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd64f9-e0be-45f0-9bb0-eff39210f29b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.385961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.386565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.386894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.387157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.387799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.389437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.403226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1263/2539] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:13.404538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.419526 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a212f3-a1b6-4298-8c79-cee414430744 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.425227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.425910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.426281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.426556 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.427239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.428858 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586a90-7cdd-4332-a1ca-6aaf7b055177 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.451879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1277/2540] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:13.456045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.468778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48378f1-fe2d-448f-bc09-11b482298647 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.475351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.476010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.476288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.476556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] PATH_INFO: `/v3/projects/f341c73c28634d669c95f7d3ffec9cf0/users/0bcc2c8ae1f944e9bb84e5bdb7cc5cb2/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.498848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f341c73c28634d669c95f7d3ffec9cf0, user_id=0bcc2c8ae1f944e9bb84e5bdb7cc5cb2, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.503185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.517559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1264/2541] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:13 2022] PUT /identity/v3/projects/f341c73c28634d669c95f7d3ffec9cf0/users/0bcc2c8ae1f944e9bb84e5bdb7cc5cb2/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:13.528112 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.528698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.528698 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.568490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] MFA Rules not processed for user `0bcc2c8ae1f944e9bb84e5bdb7cc5cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:13.570285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:13.651428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1278/2542] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:13.656293 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.697076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:13.765765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff3f3-6e98-4fb6-a1a8-114e6eb709d1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.771688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.772388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.772674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.772948 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.773592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.775618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.811802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1265/2543] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:13.815613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.827218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f7b5-d32f-4bae-b65e-d78657aea256 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.875241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:13.914965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.915633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.915914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.916217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.916856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.918876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.942474 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1279/2544] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:13 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:13.946537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.958016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411632-51f4-4170-be4e-9762e542220e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.962806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.963413 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:13.963689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:13.963960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:13.964607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:13.966250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:13.981529 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1266/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:47:13.982807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:13.993764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a3d16-aab3-4a2f-b9e7-7c8d7f6dc660 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:13.998920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:13.999669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:14.000027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:14.000382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] PATH_INFO: `/v3/projects/9edb204b73c6413d9b8df4da7256460b/users/279f3c99dd3d4b089023e2f7d724b587/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:14.017676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edb204b73c6413d9b8df4da7256460b, user_id=279f3c99dd3d4b089023e2f7d724b587, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:14.020527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:14.037490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1280/2546] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:13 2022] PUT /identity/v3/projects/9edb204b73c6413d9b8df4da7256460b/users/279f3c99dd3d4b089023e2f7d724b587/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:14.039410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:14.053141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbc271-9a3b-485c-834e-19b82e6e528e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:14.122313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:14.123074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:14.123419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:14.123778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:14.124495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:14.126248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033356a2-af2f-45cd-b5cf-6a43b83db390 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:14.140235 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1267/2547] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:14.142296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:14.153463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c10fc-54e6-4723-b8ba-d40c5ff2314c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:14.158935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:14.159602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:14.159878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:14.160148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] PATH_INFO: `/v3/projects/9edb204b73c6413d9b8df4da7256460b/users/279f3c99dd3d4b089023e2f7d724b587/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:14.162097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edb204b73c6413d9b8df4da7256460b, user_id=279f3c99dd3d4b089023e2f7d724b587, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:14.164788 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:14.176205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1281/2548] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:14 2022] PUT /identity/v3/projects/9edb204b73c6413d9b8df4da7256460b/users/279f3c99dd3d4b089023e2f7d724b587/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:14.187253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:14.187537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:14.187816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:14.219161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] MFA Rules not processed for user `279f3c99dd3d4b089023e2f7d724b587`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:14.220969 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:14.305599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1268/2549] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:14.318221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:14.320020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:14.384492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc2be-9ac5-47aa-b71c-829743ead720 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:14.396728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:14.396728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:14.396728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:14.396728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:14.457766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:14.459425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:14.470753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1282/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:18.903088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:18.912571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92508d35-cf92-4837-97d2-b83a45dc74e7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:18.916173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:18.916763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:18.917043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:18.917313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:18.926430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:18.927884 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1269b97-b78a-45ec-8c04-36a4e6af8c82 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:18.937126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1269/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:23.824317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:23.858358 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:23.886243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bd81-7788-4d23-9b46-efdc60114917 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:23.890719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:23.891872 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:23.892148 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:23.892418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] PATH_INFO: `/v3/users/0bcc2c8ae1f944e9bb84e5bdb7cc5cb2` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:23.893517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bcc2c8ae1f944e9bb84e5bdb7cc5cb2)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:23.895466 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:23.940440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bcc2c8ae1f944e9bb84e5bdb7cc5cb2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:23.941085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bcc2c8ae1f944e9bb84e5bdb7cc5cb2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:23.941683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bcc2c8ae1f944e9bb84e5bdb7cc5cb2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:23.954284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bcc2c8ae1f944e9bb84e5bdb7cc5cb2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:23.961221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bcc2c8ae1f944e9bb84e5bdb7cc5cb2'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:23.974394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1283/2552] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:23 2022] DELETE /identity/v3/users/0bcc2c8ae1f944e9bb84e5bdb7cc5cb2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:23.979312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:23.979822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:23.979930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:23.987752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:24.004107 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:24.005357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:24.066752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1270/2553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:24.076316 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:24.078182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:24.136364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6408d9-862f-4efa-b20d-7f066b8b8c2f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:24.139922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:24.140529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:24.140809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:24.141629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:24.150954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:24.152439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:24.162252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1284/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:24.551136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:24.552949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:24.567791 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b2307-6704-4820-902b-4294ee7bb4ca None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:24.574972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:24.575893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:24.576257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:24.576641 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] PATH_INFO: `/v3/projects/f341c73c28634d669c95f7d3ffec9cf0` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:24.577933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f341c73c28634d669c95f7d3ffec9cf0)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:24.579952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:24.647522 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f341c73c28634d669c95f7d3ffec9cf0', 'request_id': 'req-7848ea1d-426c-4f92-82b3-77946907ce11'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.656037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f341c73c28634d669c95f7d3ffec9cf0', 'request_id': 'req-7848ea1d-426c-4f92-82b3-77946907ce11'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.664530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f341c73c28634d669c95f7d3ffec9cf0', 'request_id': 'req-7848ea1d-426c-4f92-82b3-77946907ce11'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.675973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] The token cache is being invalidate because project f341c73c28634d669c95f7d3ffec9cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:24.676267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.676924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.678739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1271/2555] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:24 2022] DELETE /identity/v3/projects/f341c73c28634d669c95f7d3ffec9cf0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:24.682512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:24.693637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619bc1-49fe-496e-afc9-7b3bcc9c7345 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:24.746631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:24.757787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:24.758741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:24.759024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:24.759297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] PATH_INFO: `/v3/users/279f3c99dd3d4b089023e2f7d724b587` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:24.760468 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=279f3c99dd3d4b089023e2f7d724b587)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:24.762407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:24.809671 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279f3c99dd3d4b089023e2f7d724b587'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.810324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279f3c99dd3d4b089023e2f7d724b587'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.810956 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279f3c99dd3d4b089023e2f7d724b587'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.823733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279f3c99dd3d4b089023e2f7d724b587'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.831928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1f3539f9-f77b-4ec3-a558-f1d797c278a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279f3c99dd3d4b089023e2f7d724b587'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:24.843950 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1285/2556] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:24 2022] DELETE /identity/v3/users/279f3c99dd3d4b089023e2f7d724b587 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:25.121511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.123030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.136920 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ea1d-426c-4f92-82b3-77946907ce11 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.194566 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.195270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.195552 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.195826 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] PATH_INFO: `/v3/projects/9edb204b73c6413d9b8df4da7256460b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.196963 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edb204b73c6413d9b8df4da7256460b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.198907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.231031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:25.248382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edb204b73c6413d9b8df4da7256460b', 'request_id': 'req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:25.256071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edb204b73c6413d9b8df4da7256460b', 'request_id': 'req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:25.262476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edb204b73c6413d9b8df4da7256460b', 'request_id': 'req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:25.273177 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] The token cache is being invalidate because project 9edb204b73c6413d9b8df4da7256460b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:25.273463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:25.275207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72a853c0-4cdd-49d7-a859-38bbeb20b9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:25.276748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1272/2557] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:25 2022] DELETE /identity/v3/projects/9edb204b73c6413d9b8df4da7256460b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:25.282679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.283368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.284421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.323793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:25.325166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.396226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1286/2558] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:25.404500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.404824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.405220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.428930 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:25.430961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.445751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1273/2559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:25.446876 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.457822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7db5f-9365-4b88-a3aa-a031cb520bc4 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.463074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.463737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.464017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.464372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.465037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:25.465365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.467200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.481766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1287/2560] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:47:25.486074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.495542 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa357e-1b41-4e12-aeb8-007198487e37 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.503193 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.503397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.503752 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.504074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.504762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.506927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.535770 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1274/2561] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:25.540107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.551325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e970e57-a931-4506-9c81-3bd2b422a57f admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.614108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.615078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.615362 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.615750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.616387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.618683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.631087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:25.644527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1288/2562] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/users => generated 280 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:25.648571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.671546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b9fbd-abbb-497d-a249-22a29f0dae5e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.677509 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.678206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.678503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.678798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.679449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.681040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.694738 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1275/2563] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:25.696590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.707139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b64a32-7d99-41aa-9fa6-afb5a06cab5e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.711849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.712454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.712730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.712999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.713649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.715272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfd7f7-5d98-461f-a714-90caa1d67ed2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.727336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1289/2564] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:25.731821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.743987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c27f-142e-4f57-839a-de1e46890284 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.749799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.750574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.751058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.751338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] PATH_INFO: `/v3/projects/7640b94930c84fb68eba9a2e04036113/users/4ca155a0dd244fcb92c2a8ac4ce27327/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.767780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7640b94930c84fb68eba9a2e04036113, user_id=4ca155a0dd244fcb92c2a8ac4ce27327, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.770425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.786580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1276/2565] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:25 2022] PUT /identity/v3/projects/7640b94930c84fb68eba9a2e04036113/users/4ca155a0dd244fcb92c2a8ac4ce27327/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:25.798589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.798917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.799199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.827828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] MFA Rules not processed for user `4ca155a0dd244fcb92c2a8ac4ce27327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:25.829086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.894472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1290/2566] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:25.927325 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.929001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.987172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb7766-b6db-402a-83b4-04fdfed675d9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.991120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.991736 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.992198 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.992478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.001171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:26.002604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3660-abb7-4dc4-a2aa-630436540eca service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:26.011737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1277/2567] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:26.027886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:26.094146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f71eb-8bba-42f2-9f32-6d01e3fe200c None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:26.099177 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:26.099802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.100080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.100351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] PATH_INFO: `/v3/users/4ca155a0dd244fcb92c2a8ac4ce27327` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.101487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca155a0dd244fcb92c2a8ac4ce27327)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:26.103740 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:26.156731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca155a0dd244fcb92c2a8ac4ce27327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.160334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca155a0dd244fcb92c2a8ac4ce27327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.160982 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca155a0dd244fcb92c2a8ac4ce27327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.174732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca155a0dd244fcb92c2a8ac4ce27327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.183727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca155a0dd244fcb92c2a8ac4ce27327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.185538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:26.199011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1291/2568] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:26 2022] DELETE /identity/v3/users/4ca155a0dd244fcb92c2a8ac4ce27327 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:26.204034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.204533 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.204643 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.224862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:26.226087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:26.275899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:26.289076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1278/2569] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:26.299155 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:26.300587 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:26.361635 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409f0f5-4c88-438e-aa53-41c0555a6eb8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:26.365295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:26.365893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.366200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.366465 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.375883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:26.377299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c845-f08f-487e-b513-9fa89f9497be service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:26.386505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1292/2570] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:26.740447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:26.742233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:26.757113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af5bc-c997-4e8f-864a-052a788b8a2a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:26.762034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:26.762674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.762985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.763282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] PATH_INFO: `/v3/projects/7640b94930c84fb68eba9a2e04036113` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.764443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7640b94930c84fb68eba9a2e04036113)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:26.766365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:26.812648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7640b94930c84fb68eba9a2e04036113', 'request_id': 'req-03e79916-a44d-4b88-8cb2-2d46c8196a57'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.820564 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7640b94930c84fb68eba9a2e04036113', 'request_id': 'req-03e79916-a44d-4b88-8cb2-2d46c8196a57'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.826944 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7640b94930c84fb68eba9a2e04036113', 'request_id': 'req-03e79916-a44d-4b88-8cb2-2d46c8196a57'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.837501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] The token cache is being invalidate because project 7640b94930c84fb68eba9a2e04036113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:26.837792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.838455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03e79916-a44d-4b88-8cb2-2d46c8196a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:26.841309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1279/2571] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:26 2022] DELETE /identity/v3/projects/7640b94930c84fb68eba9a2e04036113 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:26.846210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.846572 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.846947 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.867096 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:26.868407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:26.930017 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1293/2572] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:26.939170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.939464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:26.939743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:26.960635 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:26.961858 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:26.971045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1280/2573] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:26.975086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:26.994499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da833d-dd3b-4c88-a0a3-af6c8786d31f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:26.999119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:26.999719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:26.999996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.000909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.001558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:27.001860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.003484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.015074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1294/2574] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:47:27.019409 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.028652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f176e-4543-40b5-bac4-d185b2e974fc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.033836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.034423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.034723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.035005 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.035634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.037541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.066000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1281/2575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:27 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:27.070124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.080135 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dea8-f73d-4f4f-92f4-62d595b1752a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.135624 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.136220 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.136500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.136776 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.137398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.139376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.160312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1295/2576] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:27 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:27.164424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.175454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae720-8071-4c94-9537-246a7fdf3ff9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.180022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.180605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.180879 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.181147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.181785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.183415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.193057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1282/2577] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:27.196723 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.206376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c189-9533-4594-9a3c-a125e49853bd admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.216152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.216866 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.217195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.217518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.218234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.220588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949b699-9c54-4a2b-b6ec-cf13f8a4cbf7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.222785 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:27.233517 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1296/2578] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:27.238120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.248115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f39e5-1205-4dbc-b162-aa82d52c2979 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.252833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.253474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.253757 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.254030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] PATH_INFO: `/v3/projects/ed06dd05a0504aaf87a8d5d129edcc98/users/d8b8896e3d264f30911c6a58e6db71c9/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.268345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed06dd05a0504aaf87a8d5d129edcc98, user_id=d8b8896e3d264f30911c6a58e6db71c9, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.271044 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.284605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1283/2579] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:27 2022] PUT /identity/v3/projects/ed06dd05a0504aaf87a8d5d129edcc98/users/d8b8896e3d264f30911c6a58e6db71c9/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:27.294734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407217-2586-434a-a7a8-60a3707e6a5b None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.295024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407217-2586-434a-a7a8-60a3707e6a5b None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.295302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407217-2586-434a-a7a8-60a3707e6a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.323875 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b0407217-2586-434a-a7a8-60a3707e6a5b None None] MFA Rules not processed for user `d8b8896e3d264f30911c6a58e6db71c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:27.325120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b0407217-2586-434a-a7a8-60a3707e6a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:27.403114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1297/2580] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:27.415453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:27.417176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:27.514369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a277-27ae-4256-af71-6cdfe305b8d2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:27.520054 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:27.520608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:27.520885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:27.521153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:27.530428 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:27.532776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:27.542125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1284/2581] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:29.288399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:29.290132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:29.290411 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:29.316558 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:29.318222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:29.392254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1298/2582] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:47:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:29.406328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:29.408195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:29.432524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:29.488457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb5b2d-1027-4420-bd71-307c4d88b352 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:29.493478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:29.494114 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:29.494389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:29.494656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:29.503817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:29.505734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acd9fd-2d31-4442-a8ac-f390fb79f1ee service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:29.514824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1285/2583] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:47:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:47:38.423001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:38.424408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:38.480546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578d29b-1a11-44a0-aadf-843efef612fc None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:38.484123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:38.485390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:38.485666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:38.485939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:38.487676 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:38.531116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:38.549515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:38.550962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69bcda-c891-4410-903e-e6ba63630047 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:38.560599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1299/2584] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:38.960585 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:38.961129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:38.961129 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:38.962480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1286/2585] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:38.969809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64d1c-132a-488e-9c91-40ad5b4e33d6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:38.970097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64d1c-132a-488e-9c91-40ad5b4e33d6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:38.970372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64d1c-132a-488e-9c91-40ad5b4e33d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:38.997523 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6fb64d1c-132a-488e-9c91-40ad5b4e33d6 None None] MFA Rules not processed for user `ae3cfe87a8b347729eb0cbb4f718d394`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:38.999200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6fb64d1c-132a-488e-9c91-40ad5b4e33d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.015166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1300/2586] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:47:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:39.025381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:39.028112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.043939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc59caa-ef78-4fb8-a970-865116f29e98 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:39.048074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:39.048761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:39.049259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:39.049611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.058435 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:39.060168 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357455d-cdb4-4eb3-8280-7efe9e11afe5 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:39.070596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1287/2587] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:39.179798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5fa7-6725-4c41-80a8-97226c766664 None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:39.180081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5fa7-6725-4c41-80a8-97226c766664 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:39.180360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5fa7-6725-4c41-80a8-97226c766664 None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.181708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1301/2588] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 24 07:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:39.186865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:39.187188 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:39.188002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.208345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] MFA Rules not processed for user `cff82f8d12d842d1baab39bd73ff54cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:39.209562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.281696 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1288/2589] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 24 07:47:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:39.292255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d5fa7-6725-4c41-80a8-97226c766664 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:39.304235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d5fa7-6725-4c41-80a8-97226c766664 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:39.308409 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:39.309617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:39.309898 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:39.310176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.321855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:39.323348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:39.335169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1302/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:42.096453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:42.097935 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:42.161803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:42.163445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:42.185447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87385-06b4-411e-9546-49bd47d5143f service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:42.190197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:42.190903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:42.191181 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:42.191452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] PATH_INFO: `/v3/users/f594c9344b2c472ba33ceef87ec88a0b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:42.192582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f594c9344b2c472ba33ceef87ec88a0b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:42.194579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcbae4-9ab0-4016-b61d-6a99ee2c9dc0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:42.195056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:42.200893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:42.201582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:42.201811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:42.202354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:42.205173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:42.217851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:42.220217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:42.232291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1289/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:42.252351 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f594c9344b2c472ba33ceef87ec88a0b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.253008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f594c9344b2c472ba33ceef87ec88a0b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.253761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f594c9344b2c472ba33ceef87ec88a0b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.270287 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f594c9344b2c472ba33ceef87ec88a0b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.280494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0c2d193d-157a-47f8-975c-68807b1d1a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f594c9344b2c472ba33ceef87ec88a0b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.294152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1303/2592] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:42 2022] DELETE /identity/v3/users/f594c9344b2c472ba33ceef87ec88a0b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:42.762531 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:42.777439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412d63d-b387-4e37-8010-b01a0edfab47 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:42.791284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:42.847903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:42.848541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:42.848828 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:42.849097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] PATH_INFO: `/v3/projects/af5acb73124c485aa87e60b4e5d4f6d4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:42.850471 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5acb73124c485aa87e60b4e5d4f6d4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:42.852414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:42.905016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5acb73124c485aa87e60b4e5d4f6d4', 'request_id': 'req-5877fc3b-b979-4325-8145-9ff0c1129220'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.916130 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5acb73124c485aa87e60b4e5d4f6d4', 'request_id': 'req-5877fc3b-b979-4325-8145-9ff0c1129220'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.929460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5acb73124c485aa87e60b4e5d4f6d4', 'request_id': 'req-5877fc3b-b979-4325-8145-9ff0c1129220'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.946864 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] The token cache is being invalidate because project af5acb73124c485aa87e60b4e5d4f6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:42.947156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.948047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5877fc3b-b979-4325-8145-9ff0c1129220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:42.953743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1290/2593] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:42 2022] DELETE /identity/v3/projects/af5acb73124c485aa87e60b4e5d4f6d4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:42.955467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:42.956007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:42.956440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:42.991757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:42.993280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:43.065631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1304/2594] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:43.073063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.073354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.073628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.097051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:43.098907 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:43.110180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1291/2595] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:43.114566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.124880 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68adf93-1b65-40f9-83d7-5457c738df24 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.130078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.130791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.131076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.131347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.131993 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:43.132292 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.133909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.146402 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1305/2596] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:47:43.151074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.164798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987936ee-ee56-4502-9c13-e449488f31d7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.169741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.170408 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.170688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.171113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.172262 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.174146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.202181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1292/2597] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:43 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:47:43.206390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.216453 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0d29f-79f8-47b9-9c44-48ad3d1ded3d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.235083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:43.274633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.275373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.275657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.275934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.276579 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.278529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.300524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1306/2598] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:43 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:43.305254 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.317060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493355-c84f-4c44-9fe9-739faa0c5fe7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.322814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.323844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.324337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.324877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.326043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.329137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.341677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1293/2599] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:43.346013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.356899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaff25d-3bb1-4b5c-9199-4eeb0fb26ee0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.362489 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.363173 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.363455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.363750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.364547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.366544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f047-d7e7-4840-9419-656e31d59923 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.379251 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1307/2600] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:43.383321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.393354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1170f9-549c-4cb4-8553-cbb02ac14cd0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.398072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.398737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.399022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.399298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] PATH_INFO: `/v3/projects/dba03b7ca9994946834ac2b70aaeda5d/users/23418237eb98437eacfaee9ddf054c03/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.413799 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba03b7ca9994946834ac2b70aaeda5d, user_id=23418237eb98437eacfaee9ddf054c03, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.416486 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.431584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1294/2601] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:43 2022] PUT /identity/v3/projects/dba03b7ca9994946834ac2b70aaeda5d/users/23418237eb98437eacfaee9ddf054c03/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:43.445722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.446783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.446783 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.481302 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] MFA Rules not processed for user `23418237eb98437eacfaee9ddf054c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:43.482818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:43.556150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1308/2602] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:43.567539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.569282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:43.621055 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:43.649998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5f562-52e0-49b4-9ff8-006a0ec111b4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.655548 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.656679 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.656953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.657222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.664745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.666310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e3837-0a2a-4bf3-9600-7d1c1707eed8 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.675823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1295/2603] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:43.879878 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:43.941674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2c77c-c3be-418e-b3d1-0277575ee5c0 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:43.946144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:43.946793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:43.947076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:43.947347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] PATH_INFO: `/v3/users/23418237eb98437eacfaee9ddf054c03` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:43.948460 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=23418237eb98437eacfaee9ddf054c03)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:43.950378 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:43.986222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:44.000304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23418237eb98437eacfaee9ddf054c03'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.001034 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23418237eb98437eacfaee9ddf054c03'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.001674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23418237eb98437eacfaee9ddf054c03'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.015336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23418237eb98437eacfaee9ddf054c03'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.023119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23418237eb98437eacfaee9ddf054c03'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.036158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1309/2604] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:43 2022] DELETE /identity/v3/users/23418237eb98437eacfaee9ddf054c03 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:44.040664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.040952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.041230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.060855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:44.062076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.123905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1296/2605] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:44.133773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.135356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.195648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266158-e418-45d9-ae3f-667cab0cf7ed admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.199345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.199953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.200232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.200535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.210953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.212422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168af17-0127-47a9-909c-e5842fba7f78 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.225570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1310/2606] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:44.518358 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.520085 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.532595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc1224-74a1-41ac-99b8-68224c52f45a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.537588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.538225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.538500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.538803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] PATH_INFO: `/v3/projects/dba03b7ca9994946834ac2b70aaeda5d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.539947 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba03b7ca9994946834ac2b70aaeda5d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.541982 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.595301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba03b7ca9994946834ac2b70aaeda5d', 'request_id': 'req-20230284-17f3-45a5-bb3b-0b5f15971e4c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.603885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba03b7ca9994946834ac2b70aaeda5d', 'request_id': 'req-20230284-17f3-45a5-bb3b-0b5f15971e4c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.611003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba03b7ca9994946834ac2b70aaeda5d', 'request_id': 'req-20230284-17f3-45a5-bb3b-0b5f15971e4c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.621088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] The token cache is being invalidate because project dba03b7ca9994946834ac2b70aaeda5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:44.621375 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.621995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-20230284-17f3-45a5-bb3b-0b5f15971e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:44.623592 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1297/2607] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:44 2022] DELETE /identity/v3/projects/dba03b7ca9994946834ac2b70aaeda5d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:44.629169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.629389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.629665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.648923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:44.650111 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.709055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1311/2608] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:44.716899 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887df06-c440-4d02-965d-f612af49a7be None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.717183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887df06-c440-4d02-965d-f612af49a7be None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.717458 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887df06-c440-4d02-965d-f612af49a7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.737076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9887df06-c440-4d02-965d-f612af49a7be None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:44.738299 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9887df06-c440-4d02-965d-f612af49a7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.746286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1298/2609] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:44.751000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.760722 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f2b84-81e3-4eac-a953-d4fe1a7a63f6 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.765229 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.765861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.766169 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.766442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.767125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:44.767427 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.769013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.780279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1312/2610] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:47:44.784418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887df06-c440-4d02-965d-f612af49a7be None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.793616 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887df06-c440-4d02-965d-f612af49a7be None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.798784 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.799366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.799640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.799905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.800517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.802396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.829015 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1299/2611] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:44.833084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.843053 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095a476-71c0-4271-9f0e-b6979ee69bc8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.881340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:44.901344 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.901991 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.902273 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.902541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.903210 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.905110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.925937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1313/2612] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:44.930020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.940889 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b26d34-d40f-4ccf-84ed-8bc91272b573 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.945589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.946179 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.946457 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.946761 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.947403 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.949001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.959245 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1300/2613] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:44.963211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:44.973123 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb8d7-a5a2-4f32-8e04-ce633f71bce6 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:44.977793 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:44.978388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.978666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.978971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.979612 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:44.981225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af130-2f8c-42aa-a7ec-c5c7bda08d6f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:44.992176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1314/2614] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:44.996199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.006771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78d7d2-e413-4189-b24e-470586b2f97e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.011396 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.012037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.012314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.012586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] PATH_INFO: `/v3/projects/ee7c2616622a413283ef083d30ddbff2/users/38f8b8f88a45408fa73289ddeba8bded/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.020867 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:45.028711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7c2616622a413283ef083d30ddbff2, user_id=38f8b8f88a45408fa73289ddeba8bded, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.032412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.045402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1301/2615] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:44 2022] PUT /identity/v3/projects/ee7c2616622a413283ef083d30ddbff2/users/38f8b8f88a45408fa73289ddeba8bded/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:45.056340 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.056623 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.056901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.084104 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] MFA Rules not processed for user `38f8b8f88a45408fa73289ddeba8bded`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:45.085321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.153206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1315/2616] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:45.158020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.220747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b298104-a339-422d-a6cb-376178cd5de8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.225608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.226545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.226545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.226915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.227559 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.229545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.261374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1302/2617] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:45 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:45.262570 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.273660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119afa9a-0fa0-4a53-9263-2f1377397f17 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.332948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.333571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.333852 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.334128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.334786 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.336717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.364781 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1316/2618] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:45 2022] POST /identity/v3/users => generated 309 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:45.368823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.380818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d2651-8aa8-498f-a100-dae1e64860d5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.385463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.386058 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.386338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.386611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.387374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.388974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.399720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1303/2619] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:45.403892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.414278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29339c5-5905-4c28-bbdb-96cf2cdab66c admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.419071 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.419728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.420012 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.420288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] PATH_INFO: `/v3/projects/0f3864d7c9654cc086cf1cca8459567d/users/3e8872ebe1214fdc8985ef30da829b18/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.436089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3864d7c9654cc086cf1cca8459567d, user_id=3e8872ebe1214fdc8985ef30da829b18, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.439347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.452806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1317/2620] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:45 2022] PUT /identity/v3/projects/0f3864d7c9654cc086cf1cca8459567d/users/3e8872ebe1214fdc8985ef30da829b18/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:45.456673 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.466823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898dbf-5f4f-4197-bd0f-a72cd356b31a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.524731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.525376 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.525659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.525931 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.526580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.528212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.537848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.538305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1304/2621] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:47:45.542244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.542686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.552919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f323-b001-463e-990a-4cb0a7306349 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.556120 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3fdb-167e-4b61-b3c8-f1126b14930a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.557848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.558485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.558811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.559086 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] PATH_INFO: `/v3/projects/0f3864d7c9654cc086cf1cca8459567d/users/3e8872ebe1214fdc8985ef30da829b18/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.560764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.561000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3864d7c9654cc086cf1cca8459567d, user_id=3e8872ebe1214fdc8985ef30da829b18, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.561386 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.561665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.561944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] PATH_INFO: `/v3/users/d8b8896e3d264f30911c6a58e6db71c9` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.563669 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b8896e3d264f30911c6a58e6db71c9)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.564303 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece98a-766f-4f03-90b0-6325ad923edd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.565575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.571166 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:45.576550 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1305/2622] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:45 2022] PUT /identity/v3/projects/0f3864d7c9654cc086cf1cca8459567d/users/3e8872ebe1214fdc8985ef30da829b18/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:45.589500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.589793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.590071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.620248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b8896e3d264f30911c6a58e6db71c9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:45.621101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] MFA Rules not processed for user `3e8872ebe1214fdc8985ef30da829b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:45.621279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b8896e3d264f30911c6a58e6db71c9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:45.621766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b8896e3d264f30911c6a58e6db71c9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:45.622341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.634931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b8896e3d264f30911c6a58e6db71c9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:45.643067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8c7d999a-4d6e-48b9-8956-9dff4e916b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b8896e3d264f30911c6a58e6db71c9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:45.655493 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1318/2623] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:45 2022] DELETE /identity/v3/users/d8b8896e3d264f30911c6a58e6db71c9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:45.657256 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:45.660079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.660365 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.660666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.680731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:45.681960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.696337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1306/2624] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:45.707147 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.708568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.742091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1319/2625] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:45.752242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.753622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.767882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d001-401f-4e45-97cd-c9dbd75e5e94 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.815970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804752b2-5792-4b77-82ea-e752af90cfa5 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.818654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.819316 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.819593 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.819693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.819871 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.820205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.820488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.820847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.829830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.831303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.841199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1320/2626] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:45.882113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.883689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.893897 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1307/2627] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:46.029960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.031964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.044503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1c6cb-f07a-44c0-bc71-bf3536de22ed service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.049909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.050536 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.050846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.051124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] PATH_INFO: `/v3/users/38f8b8f88a45408fa73289ddeba8bded` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.052278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f8b8f88a45408fa73289ddeba8bded)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.054197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.077662 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:46.107119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f8b8f88a45408fa73289ddeba8bded'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.108067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f8b8f88a45408fa73289ddeba8bded'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.109074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f8b8f88a45408fa73289ddeba8bded'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.125083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f8b8f88a45408fa73289ddeba8bded'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.128072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.129595 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.134176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-10776376-126a-4868-91e5-6cc9e8d8f067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f8b8f88a45408fa73289ddeba8bded'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.148228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1321/2628] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:46 2022] DELETE /identity/v3/users/38f8b8f88a45408fa73289ddeba8bded => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:46.153735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.154199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.154634 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.158231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809361-3e79-4f9b-8773-d4b6bed6b727 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.175828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:46.176980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.193063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:46.218500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.219336 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.220117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.220389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] PATH_INFO: `/v3/projects/ed06dd05a0504aaf87a8d5d129edcc98` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.221578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed06dd05a0504aaf87a8d5d129edcc98)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.223501 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.239359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1322/2629] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:46.249121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.250608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.269277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed06dd05a0504aaf87a8d5d129edcc98', 'request_id': 'req-ef5bf217-843a-4525-bb73-30574efaa717'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.277166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed06dd05a0504aaf87a8d5d129edcc98', 'request_id': 'req-ef5bf217-843a-4525-bb73-30574efaa717'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.284272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed06dd05a0504aaf87a8d5d129edcc98', 'request_id': 'req-ef5bf217-843a-4525-bb73-30574efaa717'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.294502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] The token cache is being invalidate because project ed06dd05a0504aaf87a8d5d129edcc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:46.294821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.295455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ef5bf217-843a-4525-bb73-30574efaa717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.297337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1308/2630] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:46 2022] DELETE /identity/v3/projects/ed06dd05a0504aaf87a8d5d129edcc98 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:46.303370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1632f6-965c-4f95-b541-4207f0142aff None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.303653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1632f6-965c-4f95-b541-4207f0142aff None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.303936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1632f6-965c-4f95-b541-4207f0142aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.311179 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9cc8-cc94-419a-bc50-47318bfbd1dd None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.323310 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5d1632f6-965c-4f95-b541-4207f0142aff None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:46.324571 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5d1632f6-965c-4f95-b541-4207f0142aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.360228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.360847 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.361126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.361407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.363231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.376397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:46.388633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1309/2631] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:46.398101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.398401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.398765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.421151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:46.422674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.433135 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1310/2632] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:46.435355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.444940 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19a56b-3369-4f47-84a8-7750a8057d0e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.448748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.450753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.450874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.451289 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.451467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.451710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.452362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:46.452664 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.454839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.461628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1323/2633] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:46.472711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1311/2634] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:47:46.477263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.487036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e78e8-ad44-41be-bf4a-324911d76fd9 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.491819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.492416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.492696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.492965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.493583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.495515 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.528450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1324/2635] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:47:46.529653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.540640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac607-0c0e-42c7-83cb-fded7d5c45ff admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.595767 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:46.612589 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.613205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.613490 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.613763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.614401 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.616452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.645294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1312/2636] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:47:46.650139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.676838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bfb3b-ff2c-42da-a257-9a46a0021488 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.682949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.683641 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.683843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.684119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.684773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.686390 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.699319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1325/2637] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:46.704066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.714366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f56bb-c59c-4fa4-9eb0-d0e81022f70b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.719233 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.720050 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.720441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.720809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.721674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.723301 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.734248 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1313/2638] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:46.738428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.748602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58149e-3f79-41b2-a981-5f4d10ce3571 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.753916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.754569 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.754910 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.755199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] PATH_INFO: `/v3/projects/ed85535cac884c8bae9fddc7dcebd3a7/users/7c7548e799984498bd7ef2f237c7214f/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.771605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed85535cac884c8bae9fddc7dcebd3a7, user_id=7c7548e799984498bd7ef2f237c7214f, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.774819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b54a0d-bb19-4c7b-b19f-81bb658a3c01 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.782418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.784719 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.792067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1326/2639] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:46 2022] PUT /identity/v3/projects/ed85535cac884c8bae9fddc7dcebd3a7/users/7c7548e799984498bd7ef2f237c7214f/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:46.806998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3318-7026-464b-a648-0d61a3121af5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:46.859826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.860106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.860377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.875839 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:46.876510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:46.876790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:46.877065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] PATH_INFO: `/v3/projects/ee7c2616622a413283ef083d30ddbff2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:46.878203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7c2616622a413283ef083d30ddbff2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:46.880887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:46.888731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] MFA Rules not processed for user `7c7548e799984498bd7ef2f237c7214f`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:46.889964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.927441 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7c2616622a413283ef083d30ddbff2', 'request_id': 'req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.936133 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7c2616622a413283ef083d30ddbff2', 'request_id': 'req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.943065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7c2616622a413283ef083d30ddbff2', 'request_id': 'req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.953326 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] The token cache is being invalidate because project ee7c2616622a413283ef083d30ddbff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:46.953618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.954273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:46.955581 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1327/2640] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:46.955989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1314/2641] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:46 2022] DELETE /identity/v3/projects/ee7c2616622a413283ef083d30ddbff2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:46.959798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.966323 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:46.967713 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:46.970429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5493a-a087-46ab-88a5-1bc7dcd28d3f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.012230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:47.031619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.032247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.032525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.032792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] PATH_INFO: `/v3/users/3e8872ebe1214fdc8985ef30da829b18` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.033914 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8872ebe1214fdc8985ef30da829b18)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.035850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.038062 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb958e-8374-4900-8d7c-76c3cfd06cd6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.042013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.042612 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.042929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.043200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.045004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:47.062286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:47.094395 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8872ebe1214fdc8985ef30da829b18'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.095090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8872ebe1214fdc8985ef30da829b18'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.095716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8872ebe1214fdc8985ef30da829b18'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.108719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8872ebe1214fdc8985ef30da829b18'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.116691 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8872ebe1214fdc8985ef30da829b18'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.120523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.121973 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f72cf-e4f7-4021-8ebc-2d1ae33c4733 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.130142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1328/2642] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:46 2022] DELETE /identity/v3/users/3e8872ebe1214fdc8985ef30da829b18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:47.131615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1315/2643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:47:47.459724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:47.463290 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:47.538754 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2678e-6ac1-4820-b01b-db3c4e0e0814 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.543630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.544257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.544531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.544814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] PATH_INFO: `/v3/projects/0f3864d7c9654cc086cf1cca8459567d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.545945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3864d7c9654cc086cf1cca8459567d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.548187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.611632 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3864d7c9654cc086cf1cca8459567d', 'request_id': 'req-8a082c27-18b8-4497-acf8-1c2fac8ba31c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.620915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3864d7c9654cc086cf1cca8459567d', 'request_id': 'req-8a082c27-18b8-4497-acf8-1c2fac8ba31c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.628401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3864d7c9654cc086cf1cca8459567d', 'request_id': 'req-8a082c27-18b8-4497-acf8-1c2fac8ba31c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.631083 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:47.643606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] The token cache is being invalidate because project 0f3864d7c9654cc086cf1cca8459567d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:47.644285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.645168 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8a082c27-18b8-4497-acf8-1c2fac8ba31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:47.650803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1329/2644] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:47 2022] DELETE /identity/v3/projects/0f3864d7c9654cc086cf1cca8459567d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:47.652228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.652693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.652798 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.675266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:47.676554 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:47.751036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1316/2645] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:47.760552 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.761639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.761926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.782860 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:47.784101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:47.794590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1330/2646] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:47.798089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:47.808217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3967152-7c9b-43ac-bdba-2d4fae253f80 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.814269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.815116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.815239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.815469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.816117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:47.816419 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.818024 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.834780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1317/2647] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:47:47.836609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:47.848495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa82e95-5f76-4690-861d-ba11a832a344 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.854107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.854888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.855286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.855564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.856203 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.858099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.886031 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1331/2648] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:47 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:47.890432 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:47.900463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4b88-572a-45af-a0d1-9a968840a436 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:47.956253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:47.956849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.957131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.957410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.958029 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:47.959992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:47.980318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1318/2649] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:47 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:47.985081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:47.995421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf3c6b-cde9-482f-ab76-b95b18df47cc admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:48.000739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:48.001324 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.001610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.001891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.002544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:48.004182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:48.014195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1332/2650] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:48.018167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:48.028217 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96dcfe-65c2-4ac7-ada7-4555aa75ddf8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:48.033371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:48.033966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.034247 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.034524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.035199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:48.036805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26716dcb-6832-45a3-a016-3f8bb9befc1b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:48.048141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1319/2651] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:48.052072 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:48.062008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0afda-e0f2-412a-8ebf-7a84cee0aa92 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:48.066655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:48.067327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.067608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.067889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] PATH_INFO: `/v3/projects/9a0d2b9d70da44d8a6e92b9f6e3cd51f/users/f6e48ddefd0a45b08d515d94f14f8674/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.082058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0d2b9d70da44d8a6e92b9f6e3cd51f, user_id=f6e48ddefd0a45b08d515d94f14f8674, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:48.084739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:48.097842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1333/2652] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:48 2022] PUT /identity/v3/projects/9a0d2b9d70da44d8a6e92b9f6e3cd51f/users/f6e48ddefd0a45b08d515d94f14f8674/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:48.102788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d8e9-da24-4065-a1aa-b0c9badb089f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.103076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d8e9-da24-4065-a1aa-b0c9badb089f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.103351 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d8e9-da24-4065-a1aa-b0c9badb089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.123022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39f9d8e9-da24-4065-a1aa-b0c9badb089f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:48.124227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39f9d8e9-da24-4065-a1aa-b0c9badb089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:48.133304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:48.183640 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1320/2653] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:48.193261 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:48.195035 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:48.256003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78604-8272-4e80-8a30-21a7b484ada2 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:48.259567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:48.260171 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.260449 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.260724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.269529 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:48.270992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:48.280336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1334/2654] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:48.922312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.922946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.922946 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.957634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] MFA Rules not processed for user `f6e48ddefd0a45b08d515d94f14f8674`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:48.958922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:49.033317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1321/2655] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:49.037670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.047807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727f5a-7947-4567-8ea5-34b154d41b15 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.052455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.053151 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.053524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.053863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.054672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.056686 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.084682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1335/2656] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:49 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:49.088847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.099568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bf8f0-5948-460a-b2d3-095f6e6af023 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.158388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.158710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.158998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.159285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.159903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.161816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.182572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1322/2657] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:49 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:49.186457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.196681 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a199-96b4-4b4c-93da-8e656b4e220e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.201225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.201819 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.202097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.202370 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.203555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.205131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.216423 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1336/2658] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:49.220981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.230788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88dd9b-5154-4688-8ee0-f48cecbe3617 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.235832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.236472 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.236745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.237013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] PATH_INFO: `/v3/projects/485c1422d2ee4e37a854292697bd5f66/users/1854d702efc743919db3be54d5b8d6d7/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.255573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485c1422d2ee4e37a854292697bd5f66, user_id=1854d702efc743919db3be54d5b8d6d7, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.258745 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.272861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1323/2659] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:49 2022] PUT /identity/v3/projects/485c1422d2ee4e37a854292697bd5f66/users/1854d702efc743919db3be54d5b8d6d7/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:49.276761 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.287448 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdec57-ddfc-411d-a066-00e86b2d9a77 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.338704 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:49.347310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.348089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.348397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.348801 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.349545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.351371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7bb70-a3f4-46b2-904e-c7b3948353ac admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.361622 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1337/2660] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:49.365766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:49.376626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42681ff5-f54d-4816-82e0-566227771043 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:49.381705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:49.382780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.383068 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.383344 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] PATH_INFO: `/v3/projects/485c1422d2ee4e37a854292697bd5f66/users/1854d702efc743919db3be54d5b8d6d7/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:49.385577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=485c1422d2ee4e37a854292697bd5f66, user_id=1854d702efc743919db3be54d5b8d6d7, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:49.388493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:49.394249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:49.400407 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1324/2661] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:49 2022] PUT /identity/v3/projects/485c1422d2ee4e37a854292697bd5f66/users/1854d702efc743919db3be54d5b8d6d7/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:47:49.994482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:49.994482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:49.994482 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:50.025609 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] MFA Rules not processed for user `1854d702efc743919db3be54d5b8d6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:50.026889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:50.110492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1338/2662] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:51.135124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:51.201690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8136e-8423-4f4a-9721-f16440bad87e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:51.206547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:51.207222 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:51.207498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:51.207771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] PATH_INFO: `/v3/users/f6e48ddefd0a45b08d515d94f14f8674` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:51.208997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e48ddefd0a45b08d515d94f14f8674)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:51.210949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:51.259119 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e48ddefd0a45b08d515d94f14f8674'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.259695 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e48ddefd0a45b08d515d94f14f8674'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.267481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e48ddefd0a45b08d515d94f14f8674'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.268126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e48ddefd0a45b08d515d94f14f8674'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.281369 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e48ddefd0a45b08d515d94f14f8674'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.294746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1325/2663] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:51 2022] DELETE /identity/v3/users/f6e48ddefd0a45b08d515d94f14f8674 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:51.471976 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:51.485843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20d083-69f9-488e-a2ba-0a5eeb16b25e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:51.548283 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:51.548999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:51.549282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:51.549560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] PATH_INFO: `/v3/projects/9a0d2b9d70da44d8a6e92b9f6e3cd51f` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:51.550725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0d2b9d70da44d8a6e92b9f6e3cd51f)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:51.552655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:51.606259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0d2b9d70da44d8a6e92b9f6e3cd51f', 'request_id': 'req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.615600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0d2b9d70da44d8a6e92b9f6e3cd51f', 'request_id': 'req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.622451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0d2b9d70da44d8a6e92b9f6e3cd51f', 'request_id': 'req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.633949 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] The token cache is being invalidate because project 9a0d2b9d70da44d8a6e92b9f6e3cd51f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:51.634346 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.635295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.636916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1339/2664] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:51 2022] DELETE /identity/v3/projects/9a0d2b9d70da44d8a6e92b9f6e3cd51f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:51.641459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:51.642905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:51.655960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8d5f-0d94-4172-a161-951b1cfca9d2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:51.715508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:51.716148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:51.716426 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:51.716698 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] PATH_INFO: `/v3/users/1854d702efc743919db3be54d5b8d6d7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:51.717822 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=1854d702efc743919db3be54d5b8d6d7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:51.720340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:51.740594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:51.768383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1854d702efc743919db3be54d5b8d6d7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.769022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1854d702efc743919db3be54d5b8d6d7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.775848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1854d702efc743919db3be54d5b8d6d7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.776526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1854d702efc743919db3be54d5b8d6d7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.790503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5635be79-1e89-4403-ba5c-ec86afc52dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1854d702efc743919db3be54d5b8d6d7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:51.804487 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1326/2665] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:47:51 2022] DELETE /identity/v3/users/1854d702efc743919db3be54d5b8d6d7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:51.996138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.001186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:52.027060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad00a2-af9f-4fbc-b2dd-3b4ee8294196 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.031356 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:52.095611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.096417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.096701 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.096970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] PATH_INFO: `/v3/projects/485c1422d2ee4e37a854292697bd5f66` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.098092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485c1422d2ee4e37a854292697bd5f66)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.100051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.152079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485c1422d2ee4e37a854292697bd5f66', 'request_id': 'req-0e4808ca-21ff-427e-9a6f-61b102336327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:52.158986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485c1422d2ee4e37a854292697bd5f66', 'request_id': 'req-0e4808ca-21ff-427e-9a6f-61b102336327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:52.165194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485c1422d2ee4e37a854292697bd5f66', 'request_id': 'req-0e4808ca-21ff-427e-9a6f-61b102336327'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:52.176978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] The token cache is being invalidate because project 485c1422d2ee4e37a854292697bd5f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:47:52.177266 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:52.177903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-0e4808ca-21ff-427e-9a6f-61b102336327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:47:52.179425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1340/2666] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:47:51 2022] DELETE /identity/v3/projects/485c1422d2ee4e37a854292697bd5f66 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:52.184600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.185088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.185196 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.204854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:52.206048 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:52.264362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1327/2667] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:47:52.272143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.272428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.272703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.292011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:52.293144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:52.301636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1341/2668] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:52.305733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.316102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdaa0e-c376-434c-b51d-f823809df2db None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.320802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.321404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.321684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.321959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.322646 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:47:52.322977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.324588 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.336769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1328/2669] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:47:52.341050 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.350224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f49d-b6de-4a62-a673-3c96bc8366ca None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.355207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.355807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.356080 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.356347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.356968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.358909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.390594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1342/2670] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:52.391746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.402053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68afcb-d271-4b51-9675-e876676a29ee admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.463213 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.463814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.464094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.464362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.464994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.466929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.489433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1329/2671] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:52.494250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.505114 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fafc0a-48e4-4b8f-8a2e-299f045c8ef9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.509760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.510360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.510644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.510948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.511619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.513233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.523103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1343/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:52.527074 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.537990 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b778dc4-8ca9-4cf7-b1fe-4abe93ac12e6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.543229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.543758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.544041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.544315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.544954 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.547207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387520d9-2edb-4f6f-b8cf-2ee6c1607bf0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.557817 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1330/2673] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:52.561773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.571862 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518903b9-8ee8-44dd-9970-ccbd441db554 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.576537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.577216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.577494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.577822 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] PATH_INFO: `/v3/projects/1b9fb387c4df4904895afbf68a64e0bf/users/fc917db44613477b92d5a25aa477c113/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.592503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9fb387c4df4904895afbf68a64e0bf, user_id=fc917db44613477b92d5a25aa477c113, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.595732 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.609861 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1344/2674] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:52 2022] PUT /identity/v3/projects/1b9fb387c4df4904895afbf68a64e0bf/users/fc917db44613477b92d5a25aa477c113/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:52.619977 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.620264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.620537 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.627697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:52.649447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] MFA Rules not processed for user `fc917db44613477b92d5a25aa477c113`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:52.650730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:52.713751 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1331/2675] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:52.718228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.752720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:52.780648 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f330-76ba-445f-9af6-f66677b87e35 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.785895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.786501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.786816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.787092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.787720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.789613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.816520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1345/2676] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:47:52.820515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.831087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0720559-7559-4370-92c5-9e2b796dbdd3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.885715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.886304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.886580 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.886883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.887503 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.889445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.910748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1332/2677] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:47:52 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:47:52.914683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.925259 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8fc7f-9afe-4c16-bd27-65877ddb7db4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.929864 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.930461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.930777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.931054 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.931695 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.933286 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.944840 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1346/2678] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:52.948545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:52.958998 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e433-e7cd-4ae8-a7ac-91114b1257db admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:52.963619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:52.964255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:52.964534 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:52.964807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] PATH_INFO: `/v3/projects/02bbf753980444d98585d1ea176aec22/users/9244e852efab4df7a19cb3535f745fd4/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:52.980989 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bbf753980444d98585d1ea176aec22, user_id=9244e852efab4df7a19cb3535f745fd4, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:52.983684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:52.997212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1333/2679] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:52 2022] PUT /identity/v3/projects/02bbf753980444d98585d1ea176aec22/users/9244e852efab4df7a19cb3535f745fd4/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:53.001224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:53.011665 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd08ee-9969-40c0-9433-683558dbdc1d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:53.067328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:53.067870 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.068149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.068422 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.069062 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:53.070650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5389-aac7-48e6-86d0-c81cbe757f18 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:53.080827 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1347/2680] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:47:53.084986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:53.096017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61df4-7c33-4bd6-9b16-68d2d92ec837 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:53.100663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:53.101306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.101584 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.101857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] PATH_INFO: `/v3/projects/02bbf753980444d98585d1ea176aec22/users/9244e852efab4df7a19cb3535f745fd4/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.103852 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bbf753980444d98585d1ea176aec22, user_id=9244e852efab4df7a19cb3535f745fd4, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:53.106515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:53.116848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1334/2681] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:47:53 2022] PUT /identity/v3/projects/02bbf753980444d98585d1ea176aec22/users/9244e852efab4df7a19cb3535f745fd4/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:47:53.128056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.128366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.128639 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.157384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] MFA Rules not processed for user `9244e852efab4df7a19cb3535f745fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:53.158619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:53.225805 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1348/2682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:47:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:53.237270 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:53.239112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:53.287105 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:53.300172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2b8a1-ee56-4ff9-bd25-043ee1e6ff1d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:53.303833 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:53.304442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.304721 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.305014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.314764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:53.316240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:53.326306 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1335/2683] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:53.408140 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:53.418846 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54e4ac-2a08-4dfc-bca2-b478622af284 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:53.423097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:53.423708 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.424021 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.424298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.452257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:53.485738 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:53.487246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2994ae-887b-4c8b-9674-bc5998bacdd8 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:53.496577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1349/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:00.287626 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:00.296337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad3858-2813-404f-8b74-257ab8750f91 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:00.300291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:00.300874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:00.301145 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:00.301412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:00.303115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:00.363152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:00.364854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:00.373690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1336/2685] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:03.492454 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=15 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:08.248381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.248980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.248980 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.268539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] MFA Rules not processed for user `760115e8fb554582a016445f9aadc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:08.269746 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.276650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1350/2686] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:08.577382 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:08.579001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.659098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b935e-4ab2-4712-90d8-918733f79489 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:08.665253 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:08.666249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.666249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.666557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] PATH_INFO: `/v3/users/7c7548e799984498bd7ef2f237c7214f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.674994 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7548e799984498bd7ef2f237c7214f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:08.678065 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:08.684777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:08.695849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b8583-cd3a-4074-bc1e-8b0b5b6baef7 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:08.700379 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:08.701202 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.701490 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.701787 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] PATH_INFO: `/v3/users/fc917db44613477b92d5a25aa477c113` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.704503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc917db44613477b92d5a25aa477c113)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:08.706954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:08.731481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7548e799984498bd7ef2f237c7214f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.732775 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7548e799984498bd7ef2f237c7214f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.743475 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7548e799984498bd7ef2f237c7214f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.744445 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7548e799984498bd7ef2f237c7214f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.758324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-72f0c021-b938-4903-b493-257938a9e925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7548e799984498bd7ef2f237c7214f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.762106 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc917db44613477b92d5a25aa477c113'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.763730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc917db44613477b92d5a25aa477c113'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.764525 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc917db44613477b92d5a25aa477c113'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.776502 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1337/2687] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:08 2022] DELETE /identity/v3/users/7c7548e799984498bd7ef2f237c7214f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:08.781371 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc917db44613477b92d5a25aa477c113'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.781710 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.781997 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.782273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.790734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-68602dc5-4ae1-48bf-b338-95b5343da705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc917db44613477b92d5a25aa477c113'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:08.791744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:08.804246 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1351/2688] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:08 2022] DELETE /identity/v3/users/fc917db44613477b92d5a25aa477c113 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:08.806526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:08.808017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.808673 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.808971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.809335 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.830636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:08.831886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.871150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1338/2689] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:08.881122 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:08.882644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.902883 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1352/2690] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:08.913560 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:08.915018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.947794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e67b9-d666-4f84-8cf7-1df8a27089eb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:08.951647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:08.952255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.953077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.953352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.963803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:08.965746 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:08.976267 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1339/2691] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:48:08.981730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326d68d-8da5-4314-b5f3-7bebfd9a5f58 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:08.989274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:08.990162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.990599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.991048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:09.005438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:09.007004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:09.018304 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1353/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:48:09.352498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:09.354041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:09.359915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:09.361328 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:09.367965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf49d2b-69e6-41c6-a95f-13f7f26cdea5 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:09.373132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:09.373756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:09.375238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:09.375238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] PATH_INFO: `/v3/projects/ed85535cac884c8bae9fddc7dcebd3a7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:09.376539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed85535cac884c8bae9fddc7dcebd3a7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:09.378430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:09.391110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36e8c0-31f0-4f0e-99bf-526b2235bcd9 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:09.397464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:09.398329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:09.398679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:09.399087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] PATH_INFO: `/v3/projects/1b9fb387c4df4904895afbf68a64e0bf` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:09.401417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9fb387c4df4904895afbf68a64e0bf)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:09.404401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:09.408620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:09.453893 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed85535cac884c8bae9fddc7dcebd3a7', 'request_id': 'req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.464385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed85535cac884c8bae9fddc7dcebd3a7', 'request_id': 'req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.470301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9fb387c4df4904895afbf68a64e0bf', 'request_id': 'req-139f42b6-11a7-4a4c-9f06-84693bdb3df1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.471795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed85535cac884c8bae9fddc7dcebd3a7', 'request_id': 'req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.487707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9fb387c4df4904895afbf68a64e0bf', 'request_id': 'req-139f42b6-11a7-4a4c-9f06-84693bdb3df1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.493821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] The token cache is being invalidate because project ed85535cac884c8bae9fddc7dcebd3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:09.494544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.494785 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.496443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9fb387c4df4904895afbf68a64e0bf', 'request_id': 'req-139f42b6-11a7-4a4c-9f06-84693bdb3df1'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.496590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1340/2693] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:09 2022] DELETE /identity/v3/projects/ed85535cac884c8bae9fddc7dcebd3a7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:09.516081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] The token cache is being invalidate because project 1b9fb387c4df4904895afbf68a64e0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:09.517020 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.517916 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.520029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1354/2694] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:09 2022] DELETE /identity/v3/projects/1b9fb387c4df4904895afbf68a64e0bf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:09.524372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:09.526190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:09.570952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:09.625835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40510-ee7d-477c-b9f9-05b3ceb7b6a1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:09.693424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:09.694137 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:09.694416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:09.694708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] PATH_INFO: `/v3/users/9244e852efab4df7a19cb3535f745fd4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:09.695967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9244e852efab4df7a19cb3535f745fd4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:09.697986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:09.757184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9244e852efab4df7a19cb3535f745fd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.757976 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9244e852efab4df7a19cb3535f745fd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.765153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9244e852efab4df7a19cb3535f745fd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.765848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9244e852efab4df7a19cb3535f745fd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.777965 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86883a3a-a32b-49b1-b02d-15b9340f3711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9244e852efab4df7a19cb3535f745fd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:09.791874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1341/2695] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:09 2022] DELETE /identity/v3/users/9244e852efab4df7a19cb3535f745fd4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:10.067175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.069208 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:10.084696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f42b6-11a7-4a4c-9f06-84693bdb3df1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.169310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.170227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.170586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.171028 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] PATH_INFO: `/v3/projects/02bbf753980444d98585d1ea176aec22` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.172971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02bbf753980444d98585d1ea176aec22)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.175459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.252813 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02bbf753980444d98585d1ea176aec22', 'request_id': 'req-623e2d10-484c-4895-841e-5b5706da3461'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:10.261829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02bbf753980444d98585d1ea176aec22', 'request_id': 'req-623e2d10-484c-4895-841e-5b5706da3461'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:10.269539 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02bbf753980444d98585d1ea176aec22', 'request_id': 'req-623e2d10-484c-4895-841e-5b5706da3461'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:10.281772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] The token cache is being invalidate because project 02bbf753980444d98585d1ea176aec22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:10.282070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:10.282751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-623e2d10-484c-4895-841e-5b5706da3461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:10.284524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1355/2696] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:10 2022] DELETE /identity/v3/projects/02bbf753980444d98585d1ea176aec22 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:10.289194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.289478 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.289756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.317701 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:10.318942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:10.377152 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:10.390362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1342/2697] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:10.398437 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.398981 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.399078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.419296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:10.420521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:10.428935 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1356/2698] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:10.433847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.445034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdbf14-366c-45c2-9496-e14a932bcc72 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.449572 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.450175 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.450449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.450740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.451386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:48:10.451680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.453772 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.465950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1343/2699] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:48:10.471863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.481514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68389d07-fb6a-4af4-bd59-3d53b8fe3831 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.486267 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.486900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.487176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.487443 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.488085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.489971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.505571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:10.523090 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1357/2700] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:48:10.526766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.537226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504d72-89a2-4eca-b32b-64775860ec0f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.605694 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.606570 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.606891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.607180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.607881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.610933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.637464 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1344/2701] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:48:10.638844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.650974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95004d85-b1e7-4236-93f9-588d9a7ba28e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.656226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.656853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.657133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.657401 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.658057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.660126 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.673773 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1358/2702] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:10.675705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.686088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859cd1-3a71-4030-8921-28766d0602b3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.692667 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.693218 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.693500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.693763 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.694400 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.696094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5a98d-4bf5-4d1e-a51b-b2b9d7fe31aa admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.709335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1345/2703] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:10.713794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:10.725611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b369a53-8e9b-4251-ae1b-ad4e93fca5e4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:10.731327 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:10.732013 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.732295 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.733200 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] PATH_INFO: `/v3/projects/c5daf7592e214c80802a54dd1995240d/users/5045dc3cd9834f92bc20e1bbdbb06e16/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.749803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5daf7592e214c80802a54dd1995240d, user_id=5045dc3cd9834f92bc20e1bbdbb06e16, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:10.753288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:10.768162 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1359/2704] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:10 2022] PUT /identity/v3/projects/c5daf7592e214c80802a54dd1995240d/users/5045dc3cd9834f92bc20e1bbdbb06e16/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:10.842463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:10.842766 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:10.843094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:10.875378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] MFA Rules not processed for user `5045dc3cd9834f92bc20e1bbdbb06e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:10.878558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:10.965010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1346/2705] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:10.969364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.037767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184275-9f3e-4712-b5ba-e49fe47c217e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.049299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.049934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.050214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.050491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.051243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.053242 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.086563 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1360/2706] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:10 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:11.087936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.099051 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f28d8-bfda-4ecc-8fe8-e6e47fad8d2c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.148933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:11.158937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.159558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.159838 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.160112 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.160744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.162714 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.187645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1347/2707] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:11 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:11.192079 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.202439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81886-665a-40a1-b9e8-963c00e425c8 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.207192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.207798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.208077 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.208349 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.208992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.210591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.222263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1361/2708] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:11.226508 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.237562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37c7d-903c-474a-b329-c607c442c84c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.242191 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.242874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.243151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.243423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] PATH_INFO: `/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11/users/25e1030a350242f4b21c328964a0457a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.259447 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb49a0b8f304491dbb3c1efc164f0e11, user_id=25e1030a350242f4b21c328964a0457a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.262121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.275450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1348/2709] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:11 2022] PUT /identity/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11/users/25e1030a350242f4b21c328964a0457a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:11.279568 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.290640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eae1d-c420-426b-81b2-d6e3392f3f95 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.360363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.361153 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.361387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.361607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.362262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.363905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e5b2-c7a1-4c88-aad1-f4028540e5be admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.378619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1362/2710] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:11.379792 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.390341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c144-5cdf-4aea-b03e-42671264a237 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.395020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.395628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.395903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.396194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] PATH_INFO: `/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11/users/25e1030a350242f4b21c328964a0457a/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.398138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb49a0b8f304491dbb3c1efc164f0e11, user_id=25e1030a350242f4b21c328964a0457a, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.400809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.415313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1349/2711] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:11 2022] PUT /identity/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11/users/25e1030a350242f4b21c328964a0457a/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:11.427040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.427040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.427040 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.457350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] MFA Rules not processed for user `25e1030a350242f4b21c328964a0457a`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:11.458596 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:11.476815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:11.537227 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1363/2712] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:11.547099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:11.549025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:11.616318 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ccc-5ba7-40bb-8db8-30b8cf241835 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:11.621372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:11.621978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:11.622430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:11.622680 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:11.633132 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:11.634599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac71ba-5549-4df1-a4c8-9161f3e59b83 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:11.646538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1350/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:12.047211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:12.155347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26341c9d-9cf1-42ea-8ebe-9307680b776f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:12.159996 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:12.160650 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:12.160957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:12.161234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] PATH_INFO: `/v3/users/5045dc3cd9834f92bc20e1bbdbb06e16` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:12.162369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5045dc3cd9834f92bc20e1bbdbb06e16)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:12.164343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:12.209758 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5045dc3cd9834f92bc20e1bbdbb06e16'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:12.210406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5045dc3cd9834f92bc20e1bbdbb06e16'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:12.211037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5045dc3cd9834f92bc20e1bbdbb06e16'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:12.222927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5045dc3cd9834f92bc20e1bbdbb06e16'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:12.230008 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5045dc3cd9834f92bc20e1bbdbb06e16'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:12.232144 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:12.245156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1364/2714] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:12 2022] DELETE /identity/v3/users/5045dc3cd9834f92bc20e1bbdbb06e16 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:12.250110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:12.250395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:12.250670 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:12.299811 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:12.301019 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:12.361192 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1351/2715] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:12.371894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:12.373382 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:12.450424 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1253d7-67b7-4709-9c7c-59653cad186e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:12.454192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:12.454837 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:12.455118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:12.455398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:12.464550 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:12.465998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:12.475591 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1365/2716] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:12.948352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:12.949932 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:12.964093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59044-2dc8-48ab-9b4a-cc8cb71dfce7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:12.971093 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:12.972416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:12.973165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:12.974379 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] PATH_INFO: `/v3/projects/c5daf7592e214c80802a54dd1995240d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:12.975422 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5daf7592e214c80802a54dd1995240d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:12.977804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:12.995305 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:13.027482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5daf7592e214c80802a54dd1995240d', 'request_id': 'req-f0fdb587-92d9-49aa-8ebe-78e68140e22d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.035633 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5daf7592e214c80802a54dd1995240d', 'request_id': 'req-f0fdb587-92d9-49aa-8ebe-78e68140e22d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.042530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5daf7592e214c80802a54dd1995240d', 'request_id': 'req-f0fdb587-92d9-49aa-8ebe-78e68140e22d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.053722 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] The token cache is being invalidate because project c5daf7592e214c80802a54dd1995240d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:13.054106 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.054832 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.056330 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1352/2717] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:12 2022] DELETE /identity/v3/projects/c5daf7592e214c80802a54dd1995240d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:48:13.060264 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:13.070602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939f2d0-0918-44af-9abd-e5d20565a977 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:13.129285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:13.129913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.130194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.130467 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] PATH_INFO: `/v3/users/25e1030a350242f4b21c328964a0457a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.131608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e1030a350242f4b21c328964a0457a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:13.133524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:13.178642 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e1030a350242f4b21c328964a0457a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.179319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e1030a350242f4b21c328964a0457a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.179894 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e1030a350242f4b21c328964a0457a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.192066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e1030a350242f4b21c328964a0457a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.199128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3a742285-ec14-4a7f-a9af-e0b90e750fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e1030a350242f4b21c328964a0457a'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.206992 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:13.212727 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1366/2718] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:13 2022] DELETE /identity/v3/users/25e1030a350242f4b21c328964a0457a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:13.487689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:13.489142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:13.507909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdb587-92d9-49aa-8ebe-78e68140e22d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:13.565239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:13.565789 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.566066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.566337 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] PATH_INFO: `/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.567482 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb49a0b8f304491dbb3c1efc164f0e11)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:13.569380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:13.615280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb49a0b8f304491dbb3c1efc164f0e11', 'request_id': 'req-37606206-03c2-40b7-a57d-3ba0bbff342a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.622540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb49a0b8f304491dbb3c1efc164f0e11', 'request_id': 'req-37606206-03c2-40b7-a57d-3ba0bbff342a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.629131 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb49a0b8f304491dbb3c1efc164f0e11', 'request_id': 'req-37606206-03c2-40b7-a57d-3ba0bbff342a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.639120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] The token cache is being invalidate because project fb49a0b8f304491dbb3c1efc164f0e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:13.639402 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.640637 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-37606206-03c2-40b7-a57d-3ba0bbff342a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:13.642101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1353/2719] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:13 2022] DELETE /identity/v3/projects/fb49a0b8f304491dbb3c1efc164f0e11 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:13.647099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.647606 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.647703 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.695829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:13.697027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:13.756279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1367/2720] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:13.763856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.764139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.764414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.784365 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:13.785579 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:13.793605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1354/2721] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:13.798089 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:13.808102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e197c8-4a97-4794-b5fd-c6e7bd94727f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:13.813172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:13.813765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.814043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.814314 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.814998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:48:13.815350 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:13.816885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:13.828974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1368/2722] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:48:13.833494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:13.843162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9805f5-26b9-48a0-8e00-b4339390e93d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:13.847869 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:13.848467 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.848855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.849052 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.849688 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:13.851650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:13.879462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1355/2723] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:13 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:13.883737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:13.893939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b6312-719a-42d4-9af9-d542d51b2758 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:13.958978 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:13.960001 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.960485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.960959 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.961994 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:13.965528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:13.997874 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1369/2724] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:13 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:13.999231 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.010536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319b456-815a-4979-b84d-a1e477495cdb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.015240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.015856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.016115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.016387 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.017021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.018610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.020047 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:14.031391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1356/2725] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:14.035099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.045330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41c70-baa5-452f-8b61-088d209a8e18 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.050757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.051392 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.051672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.051945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.052592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.054184 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650100f-0dca-4a32-889e-865c02bd463f admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.066270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1370/2726] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:48:14.070172 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.080339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe460f-3f98-4872-918a-1f3d639b33cd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.084903 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.085544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.085823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.086095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] PATH_INFO: `/v3/projects/383447b3fd8142c5b4103ae96e7988a8/users/7a575ca4d3be48b5bd27f1ce4beee924/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.100295 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=383447b3fd8142c5b4103ae96e7988a8, user_id=7a575ca4d3be48b5bd27f1ce4beee924, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.102960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.116553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1357/2727] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:14 2022] PUT /identity/v3/projects/383447b3fd8142c5b4103ae96e7988a8/users/7a575ca4d3be48b5bd27f1ce4beee924/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:14.128506 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.128791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.129068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.156078 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] MFA Rules not processed for user `7a575ca4d3be48b5bd27f1ce4beee924`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:14.157293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:14.186164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:14.225406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1371/2728] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:14.230115 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.294268 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59a44-2788-4b27-9d87-b1cdaf28ce1d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.299925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.300544 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.300825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.301098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.301847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.303807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.335438 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1358/2729] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:14 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:14.336565 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.348121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a9ca3-aab3-4a35-96ec-3a1eb62a6253 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.407832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.408451 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.408734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.409009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.409826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.411868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.433170 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1372/2730] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:14 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:14.437541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.448499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c2c1-260e-48ce-b467-69cd875ce19b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.453849 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.454630 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.455002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.455347 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.456092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.457776 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.471985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1359/2731] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:14.473131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.483372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd8e7e-222d-4132-9859-48e995a16296 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.488212 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.488865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.489141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.489415 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] PATH_INFO: `/v3/projects/4a70fc2e05e44e1b8078c21618195de4/users/22b215c97ed445a5bc630e0a0ccd34ea/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.507896 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a70fc2e05e44e1b8078c21618195de4, user_id=22b215c97ed445a5bc630e0a0ccd34ea, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.510600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.525022 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1373/2732] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:14 2022] PUT /identity/v3/projects/4a70fc2e05e44e1b8078c21618195de4/users/22b215c97ed445a5bc630e0a0ccd34ea/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:14.529004 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.540241 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aebc6d-23f8-46e1-b15c-980e2d654963 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.617532 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.618042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.618320 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.618627 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.619324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.620929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63e972-797c-4665-a2a5-e30d08e6c4eb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.631759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1360/2733] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:14.636041 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.647047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69daed-3d3b-49db-a989-aed43a0f7443 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.651774 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.652522 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.652705 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.652999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] PATH_INFO: `/v3/projects/4a70fc2e05e44e1b8078c21618195de4/users/22b215c97ed445a5bc630e0a0ccd34ea/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.654944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a70fc2e05e44e1b8078c21618195de4, user_id=22b215c97ed445a5bc630e0a0ccd34ea, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.657583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.668032 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1374/2734] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:14 2022] PUT /identity/v3/projects/4a70fc2e05e44e1b8078c21618195de4/users/22b215c97ed445a5bc630e0a0ccd34ea/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:14.679010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.679297 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.679574 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.707581 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] MFA Rules not processed for user `22b215c97ed445a5bc630e0a0ccd34ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:14.708874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:14.740436 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:14.779473 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1361/2735] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:14.790925 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:14.792633 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:14.856965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89e14-39d0-43d0-ba27-9552692f6fd3 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:14.860920 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:14.861532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:14.861814 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:14.862085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:14.871150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:14.872585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bcdf0-9877-4e3e-ba23-fd7d05bb9087 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:14.882109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1375/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:15.314918 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:15.381528 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d2970-0cb6-468a-835e-905ee221720b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:15.386078 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:15.386717 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:15.387018 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:15.387288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] PATH_INFO: `/v3/users/7a575ca4d3be48b5bd27f1ce4beee924` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:15.388412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a575ca4d3be48b5bd27f1ce4beee924)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:15.390909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:15.438762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a575ca4d3be48b5bd27f1ce4beee924'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:15.439423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a575ca4d3be48b5bd27f1ce4beee924'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:15.446514 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a575ca4d3be48b5bd27f1ce4beee924'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:15.447210 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a575ca4d3be48b5bd27f1ce4beee924'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:15.460614 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a575ca4d3be48b5bd27f1ce4beee924'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:15.474308 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1362/2737] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:15 2022] DELETE /identity/v3/users/7a575ca4d3be48b5bd27f1ce4beee924 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:15.478839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:15.479129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:15.479407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:15.497941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:15.499187 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:15.518414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:15.560927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1376/2738] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:15.570911 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:15.573124 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:15.628877 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:15.635929 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7f8e-dc2b-42eb-9125-4770ba52381c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:15.641202 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:15.641886 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:15.642181 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:15.642676 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:15.652171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:15.653617 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:15.664092 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1363/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:15.955132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:15.957178 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:15.971381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f6cf8-54bd-482a-8122-7eeb3ab66648 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:15.977047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:15.977892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:15.978541 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:15.978937 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] PATH_INFO: `/v3/projects/383447b3fd8142c5b4103ae96e7988a8` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:15.980597 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383447b3fd8142c5b4103ae96e7988a8)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:15.982508 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:16.034003 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383447b3fd8142c5b4103ae96e7988a8', 'request_id': 'req-b52bd6ee-9732-4860-8ab2-f277c020dca9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.039998 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383447b3fd8142c5b4103ae96e7988a8', 'request_id': 'req-b52bd6ee-9732-4860-8ab2-f277c020dca9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.047239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383447b3fd8142c5b4103ae96e7988a8', 'request_id': 'req-b52bd6ee-9732-4860-8ab2-f277c020dca9'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.057800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] The token cache is being invalidate because project 383447b3fd8142c5b4103ae96e7988a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:16.058081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.058724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.060201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1377/2740] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:15 2022] DELETE /identity/v3/projects/383447b3fd8142c5b4103ae96e7988a8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:16.064116 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:16.073740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48533d-8943-4557-97df-5d31ef1f20ae service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:16.131523 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:16.132169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.132433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.132715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] PATH_INFO: `/v3/users/22b215c97ed445a5bc630e0a0ccd34ea` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.133823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b215c97ed445a5bc630e0a0ccd34ea)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:16.135793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:16.181296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b215c97ed445a5bc630e0a0ccd34ea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.181853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b215c97ed445a5bc630e0a0ccd34ea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.188561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b215c97ed445a5bc630e0a0ccd34ea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.189194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b215c97ed445a5bc630e0a0ccd34ea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.200890 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f5186222-0f28-469f-b3ca-a9d0eb61b711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b215c97ed445a5bc630e0a0ccd34ea'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.214404 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1364/2741] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:16 2022] DELETE /identity/v3/users/22b215c97ed445a5bc630e0a0ccd34ea => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:16.539682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:16.541262 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:16.556494 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd6ee-9732-4860-8ab2-f277c020dca9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:16.623842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:16.625005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.625339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.625618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] PATH_INFO: `/v3/projects/4a70fc2e05e44e1b8078c21618195de4` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.626923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a70fc2e05e44e1b8078c21618195de4)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:16.629440 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:16.679789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a70fc2e05e44e1b8078c21618195de4', 'request_id': 'req-86ba8d10-a6e1-437c-a8be-630fe6f7393b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.681447 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:16.688697 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a70fc2e05e44e1b8078c21618195de4', 'request_id': 'req-86ba8d10-a6e1-437c-a8be-630fe6f7393b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.697513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a70fc2e05e44e1b8078c21618195de4', 'request_id': 'req-86ba8d10-a6e1-437c-a8be-630fe6f7393b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.709098 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] The token cache is being invalidate because project 4a70fc2e05e44e1b8078c21618195de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:16.709413 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.710051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-86ba8d10-a6e1-437c-a8be-630fe6f7393b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:16.711615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1378/2742] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:16 2022] DELETE /identity/v3/projects/4a70fc2e05e44e1b8078c21618195de4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:16.716856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.717340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.717439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.736975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:16.738189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:16.797644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1365/2743] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:16.805702 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.805989 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.806269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.830312 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:16.831628 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:16.840087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1379/2744] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:16.844578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:16.854468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7361dd-8532-4123-99e7-b30112b9965c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:16.861346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:16.861956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.862240 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.862515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.863335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:48:16.863765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:16.865439 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:16.868161 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:16.881341 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1366/2745] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:48:16.885613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:16.894830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4eb0-3cb0-4aaa-8964-75ea10dd61c3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:16.899706 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:16.900307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:16.900720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:16.901004 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:16.901640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:16.903699 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:16.931278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1380/2746] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:16.935578 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:16.946150 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f71e46-ed0c-43ab-ac6c-a46498489c6d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.001909 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.002443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.002742 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.003023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.003647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.005567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.027709 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1367/2747] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:16 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:17.031747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.042377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79406ee-3200-49e6-a79a-dc83087fd1ec admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.048176 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.048780 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.049064 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.049329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.049979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.051607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.062602 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1381/2748] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:17.066608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.077076 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d314df-42ef-47b2-9235-a1ad8b3b0e80 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.081659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.082258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.082530 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.082836 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.083511 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.085088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f890c5-d485-44de-afdf-bbd8dd113439 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.096190 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1368/2749] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:17.100425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.111719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69678dd-c37c-4c61-86e7-92f000fd3ddf admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.116545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.117268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.117726 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.118009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] PATH_INFO: `/v3/projects/a2ee7f798f4142fbad0c17a911413a50/users/346aa889342f48088afdbd66e6f12880/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.132854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ee7f798f4142fbad0c17a911413a50, user_id=346aa889342f48088afdbd66e6f12880, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.135532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.149285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1382/2750] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:17 2022] PUT /identity/v3/projects/a2ee7f798f4142fbad0c17a911413a50/users/346aa889342f48088afdbd66e6f12880/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:17.160519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.160806 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.161084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.190097 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] MFA Rules not processed for user `346aa889342f48088afdbd66e6f12880`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:17.191335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:17.256165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1369/2751] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:17.260969 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.324577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a56e0-4b90-4f19-8b32-a714d701dd4a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.329225 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.329820 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.330097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.330368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.331038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.332922 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.360108 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1383/2752] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:17 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:17.364277 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.374882 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4475-560b-4098-bb0c-e511be7cb0ab None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.432488 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.433098 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.433380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.433652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.434287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.436246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.446915 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:17.458891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1370/2753] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:17 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:17.462868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.474716 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf64392-c6dd-4a98-972a-b7978df04a3e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.479496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.480105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.480387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.480657 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.481306 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.482924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.494484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1384/2754] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:17.498608 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.511067 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0145-409a-41eb-ab96-44f940e08a87 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.515383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.516449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.516953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.517380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] PATH_INFO: `/v3/projects/ec701790f94b4c7cae4a7b99174c0837/users/99183c2d7c1f4f59bcb552bb883c782d/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.537111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec701790f94b4c7cae4a7b99174c0837, user_id=99183c2d7c1f4f59bcb552bb883c782d, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.540410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.558654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1371/2755] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:17 2022] PUT /identity/v3/projects/ec701790f94b4c7cae4a7b99174c0837/users/99183c2d7c1f4f59bcb552bb883c782d/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:17.562672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.574484 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52bd7e-d890-4345-9d74-78aaa1232950 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.622675 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:17.637011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.637630 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.637909 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.638192 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.638960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.641024 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf25420-3dc5-4d79-9ef0-f758f83831c8 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.651119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1385/2756] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:17.655242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.666206 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff84d44-e712-43c1-9f89-5f16825fca47 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.670749 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.671922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.672258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.672431 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] PATH_INFO: `/v3/projects/ec701790f94b4c7cae4a7b99174c0837/users/99183c2d7c1f4f59bcb552bb883c782d/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.674411 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec701790f94b4c7cae4a7b99174c0837, user_id=99183c2d7c1f4f59bcb552bb883c782d, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.677102 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.687244 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1372/2757] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:17 2022] PUT /identity/v3/projects/ec701790f94b4c7cae4a7b99174c0837/users/99183c2d7c1f4f59bcb552bb883c782d/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:17.697330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.697615 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.697893 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.736735 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] MFA Rules not processed for user `99183c2d7c1f4f59bcb552bb883c782d`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:17.738223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:17.814047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1386/2758] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:17.825162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:17.826933 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:17.885228 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86715b0-a59e-4116-aba5-44a5f41dd5d5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:17.888922 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:17.889524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:17.890313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:17.890594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:17.899601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:17.901041 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b2f5-299c-4b3d-976a-0891552a264f service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:17.911088 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1373/2759] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:18.488957 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:18.559951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca05d-f623-409f-bb5f-89565587ac00 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:18.564674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:18.565388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:18.565731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:18.566009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] PATH_INFO: `/v3/users/346aa889342f48088afdbd66e6f12880` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:18.567832 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=346aa889342f48088afdbd66e6f12880)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:18.569206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:18.624594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346aa889342f48088afdbd66e6f12880'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:18.625280 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346aa889342f48088afdbd66e6f12880'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:18.625907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346aa889342f48088afdbd66e6f12880'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:18.638029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346aa889342f48088afdbd66e6f12880'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:18.646561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346aa889342f48088afdbd66e6f12880'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:18.648042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:18.663241 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1387/2760] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:18 2022] DELETE /identity/v3/users/346aa889342f48088afdbd66e6f12880 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:18.669201 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:18.669492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:18.669774 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:18.691672 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:18.692417 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:18.797434 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1374/2761] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:18.813571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:18.816033 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:18.902119 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b310c01-2e34-4fcc-8412-e74ea84fa728 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:18.907299 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:18.908237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:18.908718 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:18.909206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:18.920983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:18.923438 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:18.934500 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1388/2762] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:19.410357 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:19.412759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:19.445809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4919a4-48d8-4b0e-820e-b863412f5f5e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:19.453205 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:19.453985 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:19.454454 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:19.457586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] PATH_INFO: `/v3/projects/a2ee7f798f4142fbad0c17a911413a50` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:19.458741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ee7f798f4142fbad0c17a911413a50)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:19.464824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:19.550942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ee7f798f4142fbad0c17a911413a50', 'request_id': 'req-6c8b7123-61dc-402d-bf37-aa792e3fdc89'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.562744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:19.585221 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ee7f798f4142fbad0c17a911413a50', 'request_id': 'req-6c8b7123-61dc-402d-bf37-aa792e3fdc89'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.596001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ee7f798f4142fbad0c17a911413a50', 'request_id': 'req-6c8b7123-61dc-402d-bf37-aa792e3fdc89'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.609429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] The token cache is being invalidate because project a2ee7f798f4142fbad0c17a911413a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:19.609870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.610853 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.612815 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1375/2763] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:19 2022] DELETE /identity/v3/projects/a2ee7f798f4142fbad0c17a911413a50 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:19.616756 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:19.634842 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88326e30-0fe5-422d-8968-8aa7353c3248 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:19.714531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:19.715298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:19.715684 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:19.716196 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] PATH_INFO: `/v3/users/99183c2d7c1f4f59bcb552bb883c782d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:19.717472 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99183c2d7c1f4f59bcb552bb883c782d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:19.720348 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:19.797197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99183c2d7c1f4f59bcb552bb883c782d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.798582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99183c2d7c1f4f59bcb552bb883c782d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.798903 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99183c2d7c1f4f59bcb552bb883c782d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.815844 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99183c2d7c1f4f59bcb552bb883c782d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.824812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-5d7cc2a3-85e2-48a8-a275-919f73fbad91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99183c2d7c1f4f59bcb552bb883c782d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:19.839863 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1389/2764] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:19 2022] DELETE /identity/v3/users/99183c2d7c1f4f59bcb552bb883c782d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:20.185813 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:20.188229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:20.202108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7123-61dc-402d-bf37-aa792e3fdc89 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:20.265080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:20.265803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:20.266087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:20.266394 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] PATH_INFO: `/v3/projects/ec701790f94b4c7cae4a7b99174c0837` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:20.267605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec701790f94b4c7cae4a7b99174c0837)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:20.269525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:20.321395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec701790f94b4c7cae4a7b99174c0837', 'request_id': 'req-03b6aecf-49e6-409b-9429-3df2a12f9f26'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:20.329149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec701790f94b4c7cae4a7b99174c0837', 'request_id': 'req-03b6aecf-49e6-409b-9429-3df2a12f9f26'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:20.335651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec701790f94b4c7cae4a7b99174c0837', 'request_id': 'req-03b6aecf-49e6-409b-9429-3df2a12f9f26'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:20.346386 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] The token cache is being invalidate because project ec701790f94b4c7cae4a7b99174c0837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:20.346675 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:20.347349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:20.348862 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1376/2765] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:20 2022] DELETE /identity/v3/projects/ec701790f94b4c7cae4a7b99174c0837 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:22.826421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f617-9b99-4c77-bd4c-8e93be29581d None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:22.826972 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f617-9b99-4c77-bd4c-8e93be29581d None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:22.827131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f617-9b99-4c77-bd4c-8e93be29581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:22.846167 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-ab59f617-9b99-4c77-bd4c-8e93be29581d None None] MFA Rules not processed for user `a986ecc6e2644db8be8958b1eacfd5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:22.847404 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ab59f617-9b99-4c77-bd4c-8e93be29581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:22.852298 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:22.903561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1390/2766] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:30.342842 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:32.896927 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=20 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:23.988966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:23.990378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:24.086582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6aecf-49e6-409b-9429-3df2a12f9f26 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:24.094077 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:24.094716 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:24.095010 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:24.095284 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:24.097138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:24.117519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:24.119163 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:24.130423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1377/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:31.229143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:31.230010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:31.230445 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:31.283733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:31.285795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:31.362520 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1391/2768] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:49:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:31.414737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:31.415243 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:31.504070 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ddfd-de71-4180-85d8-9efa68bcb73b service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:31.510741 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:31.511013 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:31.511298 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:31.511740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:31.522182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:31.524547 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:31.550423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1378/2769] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:49:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:49:34.332207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:34.333720 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:34.440488 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5882-a146-496b-a286-e323b482b68e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:34.444457 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:34.445101 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:34.445380 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:34.445682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:34.447625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:34.676038 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:34.677406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:34.689765 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1392/2770] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:35.967959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:35.969642 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:35.994214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e00a6c-e956-4080-8436-4f14d987a2e0 service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:36.002831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:36.004538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:36.005173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:36.005650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] PATH_INFO: `/v3/users/75bdaeec19be45388613c7e22aa009db` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:36.006993 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bdaeec19be45388613c7e22aa009db)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:36.009023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:36.074339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bdaeec19be45388613c7e22aa009db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.075388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bdaeec19be45388613c7e22aa009db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.084099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:36.087759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bdaeec19be45388613c7e22aa009db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.088747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bdaeec19be45388613c7e22aa009db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.106647 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bdaeec19be45388613c7e22aa009db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.121953 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1379/2771] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:49:35 2022] DELETE /identity/v3/users/75bdaeec19be45388613c7e22aa009db => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:49:36.342408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:36.386132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22258a85-1233-4c2c-8ac6-333147889328 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:36.416900 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=100 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:36.472430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:36.473958 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:36.474547 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:36.474951 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] PATH_INFO: `/v3/projects/c75359bcb24b4beb8de99809d2bef6fa` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:36.476136 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75359bcb24b4beb8de99809d2bef6fa)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:36.478057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:36.535387 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75359bcb24b4beb8de99809d2bef6fa', 'request_id': 'req-dcbdf5e1-8545-4933-bf9b-bd02a7383330'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.549109 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75359bcb24b4beb8de99809d2bef6fa', 'request_id': 'req-dcbdf5e1-8545-4933-bf9b-bd02a7383330'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.557057 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75359bcb24b4beb8de99809d2bef6fa', 'request_id': 'req-dcbdf5e1-8545-4933-bf9b-bd02a7383330'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.570244 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] The token cache is being invalidate because project c75359bcb24b4beb8de99809d2bef6fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:49:36.570527 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.571223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.572929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1393/2772] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:49:36 2022] DELETE /identity/v3/projects/c75359bcb24b4beb8de99809d2bef6fa => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:49:36.577561 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:36.584820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:36.605497 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707de9-c46e-4a6a-8e9b-754a6b72d693 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:36.672687 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:36.673391 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:36.673671 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:36.673959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] PATH_INFO: `/v3/users/0fbe37434f3c4df7a60387c767450505` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:36.675373 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbe37434f3c4df7a60387c767450505)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:36.677331 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:36.731656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbe37434f3c4df7a60387c767450505'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.732468 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbe37434f3c4df7a60387c767450505'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.745030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbe37434f3c4df7a60387c767450505'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.745816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbe37434f3c4df7a60387c767450505'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.760410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc52d102-46f3-4661-88cf-6bcba1bed0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbe37434f3c4df7a60387c767450505'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:36.779139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1380/2773] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:49:36 2022] DELETE /identity/v3/users/0fbe37434f3c4df7a60387c767450505 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:49:36.995442 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:36.997073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:37.018611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdf5e1-8545-4933-bf9b-bd02a7383330 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:37.107521 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:37.108617 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:37.108904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:37.109174 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] PATH_INFO: `/v3/projects/5ac9cfc464234f8aaf7391bd8462521b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:37.110341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac9cfc464234f8aaf7391bd8462521b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:37.112464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:37.176944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac9cfc464234f8aaf7391bd8462521b', 'request_id': 'req-15547273-c933-4f62-b043-09a6be20b018'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:37.179724 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:37.185689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac9cfc464234f8aaf7391bd8462521b', 'request_id': 'req-15547273-c933-4f62-b043-09a6be20b018'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:37.198807 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac9cfc464234f8aaf7391bd8462521b', 'request_id': 'req-15547273-c933-4f62-b043-09a6be20b018'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:37.213553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] The token cache is being invalidate because project 5ac9cfc464234f8aaf7391bd8462521b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81678) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:49:37.214143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:37.214926 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:49:37.226519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1394/2774] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:49:36 2022] DELETE /identity/v3/projects/5ac9cfc464234f8aaf7391bd8462521b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:49:46.775539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=62,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:47.210376 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:48.336158 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774db0-3a84-421e-b72f-73a0c00b4a4d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:48.337007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774db0-3a84-421e-b72f-73a0c00b4a4d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:48.337007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774db0-3a84-421e-b72f-73a0c00b4a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:48.360169 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-79774db0-3a84-421e-b72f-73a0c00b4a4d None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:48.362276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-79774db0-3a84-421e-b72f-73a0c00b4a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:48.465621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1381/2775] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:48.483730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:48.485583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:48.617544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547273-c933-4f62-b043-09a6be20b018 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:48.626055 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:48.627105 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:48.627464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:48.627961 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:48.643334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:48.645534 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:48.660414 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1395/2776] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:49:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:49:58.458043 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=25 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:58.654138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=32 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:03.208986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:03.209455 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:03.209567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:03.228258 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:03.229450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:03.236567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1382/2777] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:50:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:03.248359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:03.259535 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f176ced-eb28-46f8-905b-a594a3dbad12 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:03.264528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:03.265125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:03.265400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:03.266317 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:03.274307 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:03.275934 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:03.284345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1396/2778] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:50:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:50:04.588610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:04.590007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:04.646230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e9792-6372-45f1-8bc9-b3df9005f661 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:04.649824 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:04.650424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:04.650730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:04.651003 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:04.652739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:04.735173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:04.736615 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:04.757120 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1383/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:07.976680 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:07.978247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:07.990797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16335-8390-4c4a-a873-7843bee898c0 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:07.994590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:07.995223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:07.995497 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:07.995764 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:07.997498 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:08.011672 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:08.013091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:08.023938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1397/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:14.754524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=66 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:18.019217 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=12 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:18.854332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:18.864618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994757-14c7-46c0-a02c-3e2b81c9a4aa service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:18.868203 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:18.868801 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:18.869082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:18.869349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:18.871099 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:18.933677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:18.935134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:18.945346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1384/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:25.408269 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:25.409907 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:25.425180 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2dee9-1d82-4a37-ade5-5f50aaac5fed service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:25.430102 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:25.430769 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:25.431048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:25.431319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] PATH_INFO: `/v3/users/760115e8fb554582a016445f9aadc00c` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:25.432503 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=760115e8fb554582a016445f9aadc00c)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:25.434436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:25.483029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760115e8fb554582a016445f9aadc00c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.483687 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760115e8fb554582a016445f9aadc00c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.484291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760115e8fb554582a016445f9aadc00c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.497198 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760115e8fb554582a016445f9aadc00c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.505389 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760115e8fb554582a016445f9aadc00c'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.521809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1398/2782] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:50:25 2022] DELETE /identity/v3/users/760115e8fb554582a016445f9aadc00c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:25.710807 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:25.721896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bc8f-fca7-4f67-a6da-bff5f3d998d7 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:25.787397 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:25.788027 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:25.788304 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:25.788576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] PATH_INFO: `/v3/projects/e97fd16d61ef4d5ebc5f435ae8a18561` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:25.789684 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97fd16d61ef4d5ebc5f435ae8a18561)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:25.791735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:25.838266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97fd16d61ef4d5ebc5f435ae8a18561', 'request_id': 'req-704d6fbf-0c5d-4817-876d-691812d5e93b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.845755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97fd16d61ef4d5ebc5f435ae8a18561', 'request_id': 'req-704d6fbf-0c5d-4817-876d-691812d5e93b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.852125 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97fd16d61ef4d5ebc5f435ae8a18561', 'request_id': 'req-704d6fbf-0c5d-4817-876d-691812d5e93b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.862875 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] The token cache is being invalidate because project e97fd16d61ef4d5ebc5f435ae8a18561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:50:25.863165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.863803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.865292 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1385/2783] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:50:25 2022] DELETE /identity/v3/projects/e97fd16d61ef4d5ebc5f435ae8a18561 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:25.869258 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:25.870730 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:25.883899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bee16-7efb-4236-9b24-656527a26eec admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:25.939397 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:25.939933 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:25.940207 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:25.940478 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] PATH_INFO: `/v3/users/676a3a75d1b74104b9d2387a7ddbb8bc` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:25.941592 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=676a3a75d1b74104b9d2387a7ddbb8bc)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:25.943524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:25.987917 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676a3a75d1b74104b9d2387a7ddbb8bc'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.988562 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676a3a75d1b74104b9d2387a7ddbb8bc'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:25.989165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676a3a75d1b74104b9d2387a7ddbb8bc'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.001644 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676a3a75d1b74104b9d2387a7ddbb8bc'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.009408 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676a3a75d1b74104b9d2387a7ddbb8bc'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.011551 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:26.025430 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1399/2784] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:50:25 2022] DELETE /identity/v3/users/676a3a75d1b74104b9d2387a7ddbb8bc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:26.212146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:26.213847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:26.240389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6fbf-0c5d-4817-876d-691812d5e93b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:26.263625 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:26.304279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:26.304900 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:26.305273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:26.306352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] PATH_INFO: `/v3/projects/17df950364db43b393289f3f6c5175e1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:26.306733 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17df950364db43b393289f3f6c5175e1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:26.308568 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:26.356274 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17df950364db43b393289f3f6c5175e1', 'request_id': 'req-c64499e0-59a7-4b91-8881-8ee3418aaec4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.363755 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17df950364db43b393289f3f6c5175e1', 'request_id': 'req-c64499e0-59a7-4b91-8881-8ee3418aaec4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.370334 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17df950364db43b393289f3f6c5175e1', 'request_id': 'req-c64499e0-59a7-4b91-8881-8ee3418aaec4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.381053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] The token cache is being invalidate because project 17df950364db43b393289f3f6c5175e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:50:26.381340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.381975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:26.383485 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1386/2785] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:50:26 2022] DELETE /identity/v3/projects/17df950364db43b393289f3f6c5175e1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:36.022082 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:36.378033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:37.178036 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:37.179461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:37.256777 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064166-38b2-41e3-a2f3-0bd8700c73fd admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:37.326576 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:37.327282 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:37.327555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:37.327823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:37.330284 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:37.389073 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:37.390717 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:37.400518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1400/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:40.522346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:40.523880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:40.590207 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64499e0-59a7-4b91-8881-8ee3418aaec4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:40.593908 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:40.594583 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:40.594895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:40.595170 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:40.596978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:40.671453 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:40.673576 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:40.683345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1387/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:43.911649 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:43.913107 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:43.928182 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e387f4-b2e1-47f7-b7ac-e0eb1e73be07 service nova] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:43.932594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:43.933209 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:43.933495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:43.933767 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] PATH_INFO: `/v3/users/a986ecc6e2644db8be8958b1eacfd5d6` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:43.934892 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a986ecc6e2644db8be8958b1eacfd5d6)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:43.936766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:43.970076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:43.986188 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a986ecc6e2644db8be8958b1eacfd5d6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:43.987070 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a986ecc6e2644db8be8958b1eacfd5d6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:43.987855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a986ecc6e2644db8be8958b1eacfd5d6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.001798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a986ecc6e2644db8be8958b1eacfd5d6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.008984 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-38d88feb-4db1-4828-bc31-b12ff59c6419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a986ecc6e2644db8be8958b1eacfd5d6'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.020936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1401/2788] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:50:43 2022] DELETE /identity/v3/users/a986ecc6e2644db8be8958b1eacfd5d6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:44.195724 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.206214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad929911-f6ce-4422-8d96-acde6d772440 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.264195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.264830 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.265113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.265390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] PATH_INFO: `/v3/projects/337840b04c0b4135a83063dfc5dbe5f1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.267079 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=337840b04c0b4135a83063dfc5dbe5f1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.269031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.320399 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337840b04c0b4135a83063dfc5dbe5f1', 'request_id': 'req-45541766-6403-408b-b32c-1d149cfcf5ff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.323153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:44.333335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337840b04c0b4135a83063dfc5dbe5f1', 'request_id': 'req-45541766-6403-408b-b32c-1d149cfcf5ff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.340173 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337840b04c0b4135a83063dfc5dbe5f1', 'request_id': 'req-45541766-6403-408b-b32c-1d149cfcf5ff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.352378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] The token cache is being invalidate because project 337840b04c0b4135a83063dfc5dbe5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:50:44.352706 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.353324 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-45541766-6403-408b-b32c-1d149cfcf5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:50:44.354896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1388/2789] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:50:44 2022] DELETE /identity/v3/projects/337840b04c0b4135a83063dfc5dbe5f1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:44.360085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.360364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.360674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.380924 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:44.382165 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:44.445979 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1402/2790] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:44.454364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.454653 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.454967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.476223 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:44.477469 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:44.485652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1389/2791] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:44.489987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.499829 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a321d15-758a-427f-b67a-a887db2f300a None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.506559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.507235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.507577 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.507885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.508575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:50:44.508912 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.510532 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.523385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1403/2792] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:50:44.527704 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.537560 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f653d54-143a-42c6-9a12-6f9557abe44f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.542165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.542805 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.543082 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.543349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.543979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.545883 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.586466 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1390/2793] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:50:44.590750 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.601164 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc07b-bf09-45f1-89d1-65cac6f14fb9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.657986 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.658688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.658995 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.659263 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.659889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.661809 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.687733 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1404/2794] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:50:44.692416 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.703313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92e53e-6ead-42d7-a399-47067af5e7a3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.708444 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.709053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.709335 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.709611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.710261 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.711912 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.724229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1391/2795] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:50:44.728487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.738850 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f864-e0ab-44aa-a425-c6a5f31efa0e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.743501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.744132 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.744416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.744693 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.745373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.747010 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965408f-ed1a-429a-b909-ad181d84f6fa admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.758588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1405/2796] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:50:44.762765 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.773611 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe44c58-a8bd-44b7-80ef-8e6491a732fc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.778197 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.778868 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.779149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.779424 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] PATH_INFO: `/v3/projects/7b4f1d7ba81a4a5b91d121b291a19e5f/users/a96230de4d9f424880e4a5fcf8d1b551/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.794545 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4f1d7ba81a4a5b91d121b291a19e5f, user_id=a96230de4d9f424880e4a5fcf8d1b551, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:44.797515 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:44.811442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1392/2797] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:50:44 2022] PUT /identity/v3/projects/7b4f1d7ba81a4a5b91d121b291a19e5f/users/a96230de4d9f424880e4a5fcf8d1b551/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:44.822291 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.822655 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.823029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.848888 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:44.851825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] MFA Rules not processed for user `a96230de4d9f424880e4a5fcf8d1b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:44.853869 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:44.919385 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1406/2798] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:44.925302 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:44.992885 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb338-0d6b-44f2-9f84-60e5a3c0e69f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:44.997430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:44.998031 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:44.998315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:44.998587 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:44.999237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.001819 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.028769 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1393/2799] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:50:44 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:50:45.032882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.043711 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d0ee-3fdf-4eff-a846-3aef1f764c73 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.100458 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.101067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.101345 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.101613 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.102249 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.104279 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.130915 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1407/2800] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:50:45 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:50:45.132601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.143524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ff9bd-5548-46c5-9898-685664591097 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.148126 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.148723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.149000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.149271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.149906 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.151535 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.162538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1394/2801] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:50:45.166841 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.180436 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437102-42be-4106-a04b-ccbdf922d6c9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.187877 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.189491 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.189964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.190428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] PATH_INFO: `/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6/users/4870a5b185f245439d5c86812de9a5a7/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.207856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbdc218f60048c89d1e3baf2ab90ec6, user_id=4870a5b185f245439d5c86812de9a5a7, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.210510 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.224056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1408/2802] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:50:45 2022] PUT /identity/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6/users/4870a5b185f245439d5c86812de9a5a7/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:50:45.228121 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.246601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f921c0-745c-4515-9a0e-5c5a7559dcb2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.291495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:45.315271 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.315928 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.316214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.316492 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.317149 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.318793 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efa695-6db2-4455-8087-8d5c8a7d32e3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.329910 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1395/2803] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:50:45.334337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.345835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e4fd-c9ea-4a1d-b5e3-85baef86ef25 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.350631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.351330 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.351611 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.351879 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] PATH_INFO: `/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6/users/4870a5b185f245439d5c86812de9a5a7/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.363141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbdc218f60048c89d1e3baf2ab90ec6, user_id=4870a5b185f245439d5c86812de9a5a7, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.365902 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.376798 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1409/2804] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:50:45 2022] PUT /identity/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6/users/4870a5b185f245439d5c86812de9a5a7/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:50:45.388748 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.389072 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.389345 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.419465 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] MFA Rules not processed for user `4870a5b185f245439d5c86812de9a5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:45.420693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:45.493500 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1396/2805] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:50:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:45.504848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.506610 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:45.567797 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:45.589400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cf9fd-1b1f-406d-a926-a4d6beca26c0 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.593131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.593734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.594011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.594895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.604417 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.606326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.617792 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1410/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:45.696282 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:45.706981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b60970-a5d5-4948-824f-3ff82371d294 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:45.710510 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:45.711146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:45.711423 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:45.711693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:45.769950 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:45.771425 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e43c7-f552-439f-9572-e11f9c7995c3 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:45.780395 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1397/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:55.612825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=10 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:55.776352 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=72 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:24.320091 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:24.381538 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b4718-7111-4860-93c7-cc1f0a494fab service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:24.386051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:24.386668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:24.386971 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:24.387235 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] PATH_INFO: `/v3/users/a96230de4d9f424880e4a5fcf8d1b551` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:24.388343 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96230de4d9f424880e4a5fcf8d1b551)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:24.390247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:24.436544 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96230de4d9f424880e4a5fcf8d1b551'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:24.437223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96230de4d9f424880e4a5fcf8d1b551'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:24.437857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96230de4d9f424880e4a5fcf8d1b551'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:24.450487 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96230de4d9f424880e4a5fcf8d1b551'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:24.457301 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96230de4d9f424880e4a5fcf8d1b551'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:24.469808 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1411/2808] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:24 2022] DELETE /identity/v3/users/a96230de4d9f424880e4a5fcf8d1b551 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:24.474272 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:24.474827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:24.474827 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:24.494437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:24.495689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:24.554184 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1398/2809] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:24.564222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:24.566222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:24.629277 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82178-ac31-4872-b3be-b0cc58778119 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:24.632944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:24.633546 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:24.633824 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:24.634097 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:24.643047 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:24.644469 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:24.653818 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1412/2810] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:24.916632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:24.918108 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:24.940558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810d9f0-3b1b-4a33-8a67-e4324db9d12f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:24.945865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:24.946538 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:24.946840 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:24.947118 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] PATH_INFO: `/v3/projects/7b4f1d7ba81a4a5b91d121b291a19e5f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:24.948234 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4f1d7ba81a4a5b91d121b291a19e5f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:24.950146 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:24.997541 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4f1d7ba81a4a5b91d121b291a19e5f', 'request_id': 'req-3c95db83-44ee-4c94-849f-b544abcde3e4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.004927 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4f1d7ba81a4a5b91d121b291a19e5f', 'request_id': 'req-3c95db83-44ee-4c94-849f-b544abcde3e4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.011389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4f1d7ba81a4a5b91d121b291a19e5f', 'request_id': 'req-3c95db83-44ee-4c94-849f-b544abcde3e4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.022294 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] The token cache is being invalidate because project 7b4f1d7ba81a4a5b91d121b291a19e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:51:25.022577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.023459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.025269 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1399/2811] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:51:24 2022] DELETE /identity/v3/projects/7b4f1d7ba81a4a5b91d121b291a19e5f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:25.029394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.039421 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb20b8-1470-4feb-9762-b6fa88fd2a74 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.057637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:25.096683 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.097848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.098122 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.098388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] PATH_INFO: `/v3/users/4870a5b185f245439d5c86812de9a5a7` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.099524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4870a5b185f245439d5c86812de9a5a7)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.101416 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.147499 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4870a5b185f245439d5c86812de9a5a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.148154 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4870a5b185f245439d5c86812de9a5a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.148763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4870a5b185f245439d5c86812de9a5a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.160983 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4870a5b185f245439d5c86812de9a5a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.168369 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-79c6ac9e-ea1a-49b6-a9a0-c01e0124f737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4870a5b185f245439d5c86812de9a5a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.180886 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1413/2812] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:25 2022] DELETE /identity/v3/users/4870a5b185f245439d5c86812de9a5a7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:25.450517 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.452601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:25.469332 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95db83-44ee-4c94-849f-b544abcde3e4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.527266 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.527913 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.528101 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.528350 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] PATH_INFO: `/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.529451 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbdc218f60048c89d1e3baf2ab90ec6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.531381 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.562634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:25.580176 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbdc218f60048c89d1e3baf2ab90ec6', 'request_id': 'req-e88645e4-f854-41c6-aea3-3aa0a74d4fed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.588658 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbdc218f60048c89d1e3baf2ab90ec6', 'request_id': 'req-e88645e4-f854-41c6-aea3-3aa0a74d4fed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.595066 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbdc218f60048c89d1e3baf2ab90ec6', 'request_id': 'req-e88645e4-f854-41c6-aea3-3aa0a74d4fed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.605881 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] The token cache is being invalidate because project cdbdc218f60048c89d1e3baf2ab90ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:51:25.606166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.606834 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e88645e4-f854-41c6-aea3-3aa0a74d4fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:25.608338 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1400/2813] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:51:25 2022] DELETE /identity/v3/projects/cdbdc218f60048c89d1e3baf2ab90ec6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:25.613751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abc821-e7cd-4221-aced-217548356a0e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.614226 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abc821-e7cd-4221-aced-217548356a0e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.614334 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abc821-e7cd-4221-aced-217548356a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.633084 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-90abc821-e7cd-4221-aced-217548356a0e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:25.634326 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-90abc821-e7cd-4221-aced-217548356a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:25.694257 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1414/2814] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:25.703237 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.703524 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.703891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.724238 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:25.725442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:25.733276 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1401/2815] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:25.737512 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abc821-e7cd-4221-aced-217548356a0e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.747719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abc821-e7cd-4221-aced-217548356a0e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.752337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.752928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.753223 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.753492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.754138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:25.754441 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.756048 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.767211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1415/2816] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:25.771891 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.781230 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b202f-46b0-4baf-bfd4-fafd1d5fa9d1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.786309 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.786925 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.787199 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.787470 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.788095 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.789986 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.818739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1402/2817] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:25 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:25.822855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.833159 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9366a-3a37-47dc-b6d8-fc3f3cdd272e admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.859364 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:25.890654 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.891275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.891559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.891828 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.892463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.894960 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.916297 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1416/2818] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:25 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:25.920493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.930894 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1522-bfc4-4625-815a-bd6d1972f9fc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.935448 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.936182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.936460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.936732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.937366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.938987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.948936 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1403/2819] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:25.952725 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.963195 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a9d8-aa2a-42a3-95b2-50cca6efdfb1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:25.967778 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:25.968372 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:25.968653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:25.968945 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:25.969582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:25.971206 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b47fc1-6356-4f31-b30d-b638dfe9a174 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:25.982045 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1417/2820] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:25.986057 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:25.995979 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea16a-7ba3-4476-a901-7435f9202c80 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.000549 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.001185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.001462 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.001732 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] PATH_INFO: `/v3/projects/cddfed54750a4e96a2c5985a3c5a5ed7/users/09e003617e614a8f9248149aafa04e63/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.015949 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cddfed54750a4e96a2c5985a3c5a5ed7, user_id=09e003617e614a8f9248149aafa04e63, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.018582 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.032180 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1404/2821] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:25 2022] PUT /identity/v3/projects/cddfed54750a4e96a2c5985a3c5a5ed7/users/09e003617e614a8f9248149aafa04e63/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:51:26.042929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.043234 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.043505 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.070121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] MFA Rules not processed for user `09e003617e614a8f9248149aafa04e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:26.071347 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:26.135885 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1418/2822] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:26.140366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.200847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c9bf-9635-48a5-a452-ea523a53b70f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.205539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.206134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.206984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.207259 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.207865 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.209760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.235183 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1405/2823] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:26 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:26.239333 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.249775 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0c49-4e74-48a0-b840-274c7dc8f7f3 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.304199 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.304723 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.304987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.305255 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.305873 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.307804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.328741 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1419/2824] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:26 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:26.333477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.343794 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c9eb-9332-4160-bd8a-4dae0125ada9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.348349 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.348937 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.349212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.349481 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.350113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.351731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.363389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1406/2825] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:26.367867 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.378637 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178ba43-4c68-49f4-8754-dd3f9198aeee admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.383288 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.383928 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.384201 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.384470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] PATH_INFO: `/v3/projects/4e656513340340a58e5226bdfc7f8990/users/f48e40f520614c588f59543e7e3e532b/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.399337 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e656513340340a58e5226bdfc7f8990, user_id=f48e40f520614c588f59543e7e3e532b, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.401964 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.414590 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1420/2826] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:26 2022] PUT /identity/v3/projects/4e656513340340a58e5226bdfc7f8990/users/f48e40f520614c588f59543e7e3e532b/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:51:26.418526 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.428960 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde99d4-1e3b-42cd-9a97-232862adce7d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.448984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:26.487287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.488476 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.488809 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.489023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.489656 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.491273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa2e89-1973-4772-aa85-00668a7c90e2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.501006 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1407/2827] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:51:26.505147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.510674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:26.516619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15288-4381-4474-9ec9-c32dd5670a76 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.522103 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.522789 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.523329 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.523463 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] PATH_INFO: `/v3/projects/4e656513340340a58e5226bdfc7f8990/users/f48e40f520614c588f59543e7e3e532b/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.525406 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e656513340340a58e5226bdfc7f8990, user_id=f48e40f520614c588f59543e7e3e532b, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.528085 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.539260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1421/2828] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:26 2022] PUT /identity/v3/projects/4e656513340340a58e5226bdfc7f8990/users/f48e40f520614c588f59543e7e3e532b/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:26.549821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.550288 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.550437 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.579113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] MFA Rules not processed for user `f48e40f520614c588f59543e7e3e532b`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:26.580255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:26.650186 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1408/2829] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:26.661941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.664014 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:26.723118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb84e-7321-4898-804a-0d8c4174dd81 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.726815 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.727425 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.727701 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.727974 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.736936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.738383 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.747999 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1422/2830] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:26.822314 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:26.832648 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75aba-9398-4fc7-91dc-d9a73a0e8c21 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:26.836061 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:26.836652 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:26.836926 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:26.837195 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:26.894286 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:26.895924 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15020f3-4113-4673-be7c-6d9c70c91ee5 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:26.905141 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1409/2831] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:36.037250 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:36.097459 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb56a3-0627-41fe-b201-191ebd8fa8b8 service cinder] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:36.102310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:36.102970 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:36.103253 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:36.103528 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] PATH_INFO: `/v3/users/09e003617e614a8f9248149aafa04e63` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:36.104656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e003617e614a8f9248149aafa04e63)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:36.106583 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:36.152461 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e003617e614a8f9248149aafa04e63'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.153030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e003617e614a8f9248149aafa04e63'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.153631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e003617e614a8f9248149aafa04e63'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.165186 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e003617e614a8f9248149aafa04e63'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.172806 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e003617e614a8f9248149aafa04e63'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.185002 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1423/2832] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:36 2022] DELETE /identity/v3/users/09e003617e614a8f9248149aafa04e63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:36.189558 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:36.190037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:36.190151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:36.209905 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:36.211104 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:36.250080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:36.269117 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1410/2833] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:36.279452 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:36.280843 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:36.294211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:36.338938 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038658-c22f-475c-91cb-27ef47ab13e4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:36.342719 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:36.343341 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:36.343619 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:36.343943 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:36.352936 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:36.354363 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:36.364331 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1424/2834] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:51:36.626623 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:36.628151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:36.643232 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729689e8-e7bb-4378-93d7-62986bcf5dfa None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:36.649075 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:36.650291 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:36.650449 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:36.650743 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] PATH_INFO: `/v3/projects/cddfed54750a4e96a2c5985a3c5a5ed7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:36.651984 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cddfed54750a4e96a2c5985a3c5a5ed7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:36.653780 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:36.700764 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cddfed54750a4e96a2c5985a3c5a5ed7', 'request_id': 'req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.707553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cddfed54750a4e96a2c5985a3c5a5ed7', 'request_id': 'req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.714279 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cddfed54750a4e96a2c5985a3c5a5ed7', 'request_id': 'req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.725239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] The token cache is being invalidate because project cddfed54750a4e96a2c5985a3c5a5ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:51:36.725525 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.726167 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.727728 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1411/2835] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:51:36 2022] DELETE /identity/v3/projects/cddfed54750a4e96a2c5985a3c5a5ed7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:36.731849 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:36.742274 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f4bd-6671-4d61-bb08-1b4276cd17a3 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:36.798483 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:36.799141 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:36.799418 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:36.799689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] PATH_INFO: `/v3/users/f48e40f520614c588f59543e7e3e532b` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:36.800823 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48e40f520614c588f59543e7e3e532b)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:36.802747 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:36.848605 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48e40f520614c588f59543e7e3e532b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.849293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48e40f520614c588f59543e7e3e532b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.849895 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48e40f520614c588f59543e7e3e532b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.861723 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48e40f520614c588f59543e7e3e532b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.869066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-f8191966-ad40-47c6-b5bc-b1fb39fdf3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48e40f520614c588f59543e7e3e532b'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:36.881713 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1425/2836] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:36 2022] DELETE /identity/v3/users/f48e40f520614c588f59543e7e3e532b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:37.155084 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.156565 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:37.172715 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8f4bf-5664-4eb7-b3f8-1ad7d96e7703 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.232287 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.232972 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.233109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.233380 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] PATH_INFO: `/v3/projects/4e656513340340a58e5226bdfc7f8990` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.234594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e656513340340a58e5226bdfc7f8990)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.236562 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.282995 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e656513340340a58e5226bdfc7f8990', 'request_id': 'req-e86f9f16-a942-4438-8375-beb581befba8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:37.290880 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e656513340340a58e5226bdfc7f8990', 'request_id': 'req-e86f9f16-a942-4438-8375-beb581befba8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:37.298200 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e656513340340a58e5226bdfc7f8990', 'request_id': 'req-e86f9f16-a942-4438-8375-beb581befba8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:37.305220 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:37.310226 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] The token cache is being invalidate because project 4e656513340340a58e5226bdfc7f8990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:51:37.310861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:37.311711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e86f9f16-a942-4438-8375-beb581befba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:37.313697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1412/2837] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:51:37 2022] DELETE /identity/v3/projects/4e656513340340a58e5226bdfc7f8990 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:37.320138 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.320689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.320689 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.340026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:37.341232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:37.370015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:37.401336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1426/2838] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:37.462212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.462495 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.462803 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.483001 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:37.484134 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:37.492596 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1413/2839] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:37.496803 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.506475 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277b219-169d-43d4-85a8-80d7c06b3cba None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.511074 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.511668 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.511948 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.512228 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.512891 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:37.513216 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.514835 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.526218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1427/2840] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:37.530607 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.542390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf821cb-046b-4f64-9887-47529bd57529 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.547113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.547711 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.548250 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.548518 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.549143 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.551739 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.578987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1414/2841] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:37 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:37.583646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.594043 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32197c8-9cb8-460e-8fb1-06b52c0c8976 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.649636 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.650238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.650516 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.650825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.651455 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.653428 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.674585 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1428/2842] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:37 2022] POST /identity/v3/users => generated 279 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:51:37.680848 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.691214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b9d4a-5e72-4536-870b-105ed7297839 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.696142 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.696737 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.697016 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.697285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.697919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.699618 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.709599 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1415/2843] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:37.713594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.724065 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37276870-bbfb-42a9-a65e-d5d8dd277ee1 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.728509 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.729094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.729366 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.729631 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.730265 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.731857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c726335-d61f-49af-9305-1ab215955707 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.742762 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1429/2844] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:37.746762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.757138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb042ec-5c20-4822-b8c6-8eb8393450de admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.761956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.762634 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.762981 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.763255 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] PATH_INFO: `/v3/projects/547d5b893f4d4b2d979f8598fefb15a8/users/43b5981a73e047e0b401b8912a66d2a7/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.779744 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] RBAC: Authorizing `identity:create_grant(project_id=547d5b893f4d4b2d979f8598fefb15a8, user_id=43b5981a73e047e0b401b8912a66d2a7, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.782384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.796959 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1416/2845] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:37 2022] PUT /identity/v3/projects/547d5b893f4d4b2d979f8598fefb15a8/users/43b5981a73e047e0b401b8912a66d2a7/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:37.807015 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.807293 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.807561 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.835018 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] MFA Rules not processed for user `43b5981a73e047e0b401b8912a66d2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:37.836214 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:37.899712 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1430/2846] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:51:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:37.911023 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:37.912703 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:37.974056 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4040d-f89d-40f1-b174-6948678d390e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:37.977747 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:37.978371 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:37.978650 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:37.978956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:37.987389 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:37.989346 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d27e3-5b1f-4948-8585-8d8d5e1bfaee service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:37.998166 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1417/2847] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:47.896889 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:47.994110 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:10.925429 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:10.987905 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8626e0-56ca-4207-912f-5fd3a71994fa None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:10.993023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:10.993653 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:10.993932 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:10.994205 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] PATH_INFO: `/v3/users/43b5981a73e047e0b401b8912a66d2a7` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:10.995353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b5981a73e047e0b401b8912a66d2a7)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:10.997270 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.045360 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b5981a73e047e0b401b8912a66d2a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.046027 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b5981a73e047e0b401b8912a66d2a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.046721 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b5981a73e047e0b401b8912a66d2a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.059419 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b5981a73e047e0b401b8912a66d2a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.067308 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b5981a73e047e0b401b8912a66d2a7'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.080237 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1431/2848] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:10 2022] DELETE /identity/v3/users/43b5981a73e047e0b401b8912a66d2a7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:11.085674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.086239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.086239 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.105368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:11.106605 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.166754 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1418/2849] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:11.177130 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.179125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.236748 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28fd9f-fdd7-49dc-9e83-a87d03c0c1a4 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.240439 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.241554 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.241831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.242099 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.251743 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.253194 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171b4f-fa3a-469b-a642-10444997b317 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.263941 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1432/2850] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:11.540735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.542321 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.554887 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb16396-dc65-45db-9d36-2f5585fa951f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.560689 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.561340 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.561621 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.561896 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] PATH_INFO: `/v3/projects/547d5b893f4d4b2d979f8598fefb15a8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.563157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547d5b893f4d4b2d979f8598fefb15a8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.565242 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.620374 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547d5b893f4d4b2d979f8598fefb15a8', 'request_id': 'req-dbcb77a0-9573-4bcf-9361-798363167343'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.628319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547d5b893f4d4b2d979f8598fefb15a8', 'request_id': 'req-dbcb77a0-9573-4bcf-9361-798363167343'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.635390 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547d5b893f4d4b2d979f8598fefb15a8', 'request_id': 'req-dbcb77a0-9573-4bcf-9361-798363167343'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.645823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] The token cache is being invalidate because project 547d5b893f4d4b2d979f8598fefb15a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:11.646109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.646795 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dbcb77a0-9573-4bcf-9361-798363167343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.648846 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1419/2851] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:11 2022] DELETE /identity/v3/projects/547d5b893f4d4b2d979f8598fefb15a8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:11.653812 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.654125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.654398 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.674968 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:11.676121 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.693601 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:11.738143 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1433/2852] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:11.746071 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.746368 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.746631 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.767296 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:11.768519 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.777212 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1420/2853] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:11.781514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.791464 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165c11f-8f91-4ef4-87a3-fa70b3f1d00e None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.795973 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.796574 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.796848 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.797116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.797766 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:11.798068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.799965 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.812139 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1434/2854] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:11.816553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.827153 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0380e-ea05-4a63-af96-0b433c5eb8c1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.831966 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.832577 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.832856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.833128 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.833756 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.836264 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.864033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1421/2855] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:11 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:11.869068 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.879197 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65000-fc62-4008-b197-107f8facc7f7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.937042 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.937709 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.937838 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.938069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.938728 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.940646 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.963608 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1435/2856] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:11 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:11.967820 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.978246 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4dce-3bfd-42e6-b71d-88e92a38aaff admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.983148 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.983760 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.984040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.984313 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.984967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.986720 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.997870 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1422/2857] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:12.002026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.013339 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea32d7-2126-4f44-96ca-c8c66bf37846 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.018247 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.018899 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.019189 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.019462 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.020147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.021784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e96312-25ae-4a63-a972-d2b237f38d39 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.033026 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1436/2858] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:12.037494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.047844 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e22f1-5934-4c43-9502-0bbbc5d7cf5c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.053730 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.054430 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.054740 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.055021 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] PATH_INFO: `/v3/projects/9660ab6dcc0245918507a224b838103f/users/01a15b62b0924c03ba6b388f914c140d/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.071942 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9660ab6dcc0245918507a224b838103f, user_id=01a15b62b0924c03ba6b388f914c140d, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.074613 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.088674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1423/2859] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:12 2022] PUT /identity/v3/projects/9660ab6dcc0245918507a224b838103f/users/01a15b62b0924c03ba6b388f914c140d/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:12.099019 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.099305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.099582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.128802 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] MFA Rules not processed for user `01a15b62b0924c03ba6b388f914c140d`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:12.130049 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.197929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1437/2860] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:12.209725 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.211429 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.233674 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:12.272139 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4ac09-e74b-4663-841c-a640babd5537 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.275802 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '923067fd0c864ca9bb202ee9585062f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.276418 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.276697 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.276971 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.285892 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.287392 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b75bb4-5655-4c01-a62d-cb6374e43a59 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.297162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1424/2861] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:19.579514 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:19.600871 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:19.657156 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfe52c-7ba6-472e-abd5-39dbccaa9d99 None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:19.662865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:19.663519 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.663794 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.664066 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] PATH_INFO: `/v3/users/01a15b62b0924c03ba6b388f914c140d` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.665232 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a15b62b0924c03ba6b388f914c140d)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:19.667303 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:19.721218 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a15b62b0924c03ba6b388f914c140d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:19.721908 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a15b62b0924c03ba6b388f914c140d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:19.722545 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a15b62b0924c03ba6b388f914c140d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:19.739268 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a15b62b0924c03ba6b388f914c140d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:19.747067 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a15b62b0924c03ba6b388f914c140d'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:19.763420 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1438/2862] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:19 2022] DELETE /identity/v3/users/01a15b62b0924c03ba6b388f914c140d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:19.768821 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.769378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.769378 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.789539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:19.790823 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:19.856723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1425/2863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:19.868081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:19.869600 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:19.937254 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e377-795b-417a-86e4-e12df2e1a8ac admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:19.941069 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:19.941688 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.941966 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.942239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.952222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:19.953666 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8dd28-1534-46d5-a71c-eff31c2cf9c3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:19.963696 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1439/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:20.257214 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.259087 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.273888 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37611d2c-aff1-429b-a3ad-fde332c04391 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.279025 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.279705 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.280162 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.280315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] PATH_INFO: `/v3/projects/9660ab6dcc0245918507a224b838103f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.281420 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9660ab6dcc0245918507a224b838103f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.283433 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.338036 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9660ab6dcc0245918507a224b838103f', 'request_id': 'req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.346477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9660ab6dcc0245918507a224b838103f', 'request_id': 'req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.353063 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9660ab6dcc0245918507a224b838103f', 'request_id': 'req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.364273 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] The token cache is being invalidate because project 9660ab6dcc0245918507a224b838103f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:20.364546 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.365042 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23a3ab56-9b6f-49ec-9e1f-42a54cc89842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.367111 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1426/2865] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:20 2022] DELETE /identity/v3/projects/9660ab6dcc0245918507a224b838103f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:20.372471 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.372753 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.373029 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.394368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:20.396129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.434373 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:20.467238 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1440/2866] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:20.476388 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.476804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.477194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.501693 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:20.503328 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.513227 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1427/2867] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:20.517779 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.529501 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931df94-b122-4565-8913-9c0ae4edf67f None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.535319 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.535944 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.536224 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.536496 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] PATH_INFO: `/v3/domains` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.537158 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81678) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:20.537967 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.539620 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.554056 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1441/2868] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:52:20.558856 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.568659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6a2d5-8e5c-41b0-836e-eb6d3a649599 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.573645 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.574263 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.574540 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.574847 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.575494 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.577415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.610450 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1428/2869] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:20 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:20.615023 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.626146 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f759276-98c1-4966-bddd-ed03c49e780d admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.685627 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.686275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.686559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.686868 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.687518 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.689485 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.713791 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1442/2870] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:20 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:20.718040 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.728632 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cdfe0-aa05-4b42-b398-627bd1493c6f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.733249 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.733855 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.734136 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.734410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.735091 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.736708 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.747499 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1429/2871] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:20.751660 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.762839 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94915-a241-40a2-b398-48d95d83d7e9 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.767594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.768213 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.768480 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.768760 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.769407 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.771051 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fc52a-38f0-4c98-9489-48d129dd9e2c admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.782836 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1443/2872] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:20.787312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.806974 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e358-ce77-408b-8600-826c903c8eeb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.819053 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.819734 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.820012 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.820285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] PATH_INFO: `/v3/projects/f6a8a466e6a44ceb989ba40e429c625e/users/48f23faea87840ecbf5b75753eb1d139/roles/c67e3b126ba2441aa233b7fc97e7fee1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.822825 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:20.838225 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a8a466e6a44ceb989ba40e429c625e, user_id=48f23faea87840ecbf5b75753eb1d139, role_id=c67e3b126ba2441aa233b7fc97e7fee1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.840952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.856991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1430/2873] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:20 2022] PUT /identity/v3/projects/f6a8a466e6a44ceb989ba40e429c625e/users/48f23faea87840ecbf5b75753eb1d139/roles/c67e3b126ba2441aa233b7fc97e7fee1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:20.861450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dcfc9-82d6-4a2c-b241-b2e445e2548a None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.861734 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dcfc9-82d6-4a2c-b241-b2e445e2548a None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.862011 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dcfc9-82d6-4a2c-b241-b2e445e2548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.882087 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-d21dcfc9-82d6-4a2c-b241-b2e445e2548a None None] MFA Rules not processed for user `11b0f239f5044e6880117c0daf3fe327`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:20.883674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-d21dcfc9-82d6-4a2c-b241-b2e445e2548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.948589 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1444/2874] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:20.958628 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.960980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.023113 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730d11-796d-4e9d-929d-f2e1debdc100 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:21.026678 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:21.027317 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.027590 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.027860 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.037339 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:21.039370 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:21.049489 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1431/2875] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:25.274005 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.274507 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.274607 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.310075 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.auth.core [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] MFA Rules not processed for user `48f23faea87840ecbf5b75753eb1d139`. Rule list: `[]` (Enabled: `True`). {{(pid=81678) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:25.311548 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.354133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:25.388856 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1445/2876] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:25.393440 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.404601 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a26b-406e-487c-af40-2685ca3b7206 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.409410 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.410033 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.410312 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.410586 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.411280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.413781 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.445987 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1432/2877] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:25.448094 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.460116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee729-1a5e-429c-9913-ff6ee8526dde None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.522682 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.523377 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.523652 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.523923 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.524674 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.526629 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.550260 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1446/2878] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:25.555014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.566189 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11e7f-5e5f-4598-b871-dedbb1d3580e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.572443 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.573456 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.573808 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.574156 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.575000 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.577014 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.608034 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1433/2879] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:25.612150 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.623239 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125ffd7-d6cd-4988-adb2-d70d9d8deeb7 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.628117 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.628800 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.629081 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.629359 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] PATH_INFO: `/v3/projects/c68ccf9133ee449fb83c25e518a8136d/users/17783e3179e745f4953fbe50d72094cd/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.645904 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68ccf9133ee449fb83c25e518a8136d, user_id=17783e3179e745f4953fbe50d72094cd, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.649222 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.664772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1447/2880] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:25 2022] PUT /identity/v3/projects/c68ccf9133ee449fb83c25e518a8136d/users/17783e3179e745f4953fbe50d72094cd/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:25.669002 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.679857 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e85c6-e3fc-4e56-a364-9d0be902e869 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.748327 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.748952 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.749229 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.749539 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.750194 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.751854 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782bfc6-c1b8-4f20-8f26-cc686e5c550d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.764285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1434/2881] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:25.768564 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.779857 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f74293-27f8-4469-b361-7cfda09ee8ea admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.784853 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.785586 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.785939 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.786281 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] PATH_INFO: `/v3/projects/c68ccf9133ee449fb83c25e518a8136d/users/17783e3179e745f4953fbe50d72094cd/roles/7b7098a7aed4412faff0e2ec081a365a` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.788368 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68ccf9133ee449fb83c25e518a8136d, user_id=17783e3179e745f4953fbe50d72094cd, role_id=7b7098a7aed4412faff0e2ec081a365a)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.791128 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.802678 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1448/2882] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:25 2022] PUT /identity/v3/projects/c68ccf9133ee449fb83c25e518a8136d/users/17783e3179e745f4953fbe50d72094cd/roles/7b7098a7aed4412faff0e2ec081a365a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:30.713600 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f240f-4b61-4c98-a581-63ddfca535e7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:30.714354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f240f-4b61-4c98-a581-63ddfca535e7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:30.714354 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f240f-4b61-4c98-a581-63ddfca535e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:30.745644 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-474f240f-4b61-4c98-a581-63ddfca535e7 None None] MFA Rules not processed for user `17783e3179e745f4953fbe50d72094cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:30.746895 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-474f240f-4b61-4c98-a581-63ddfca535e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:30.800762 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:30.854659 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1435/2883] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:30.863816 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:30.865784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:30.977715 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e3fe5-bd76-4a39-a858-f0e8872b4817 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:30.981763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea288fe3842541999821bd863ca9d0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:30.982399 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:30.982679 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:30.983000 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.014305 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:31.066582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:31.068310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:31.079826 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1449/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:31.233790 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e39fa4-0d80-48ca-b05e-cc51ede66775 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:31.234498 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e39fa4-0d80-48ca-b05e-cc51ede66775 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:31.234861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e39fa4-0d80-48ca-b05e-cc51ede66775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.294109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-43e39fa4-0d80-48ca-b05e-cc51ede66775 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:31.295383 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-43e39fa4-0d80-48ca-b05e-cc51ede66775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:31.372366 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1436/2885] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:52:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:31.426594 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:31.428929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:31.537851 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07370653-d3fd-4a6b-8834-8cb0187e826b service glance] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:31.544296 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:31.545271 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:31.545751 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:31.546231 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.562086 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:31.565052 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:31.580470 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1450/2886] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:52:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:52:40.421939 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.422731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.422731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.442557 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] MFA Rules not processed for user `48f23faea87840ecbf5b75753eb1d139`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:40.444412 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.451442 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1437/2887] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 24 07:52:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:40.719149 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.780092 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f9169-4521-4cc4-a457-d1b027232625 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.783865 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.784476 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.784757 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.785030 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.786830 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.797571 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.799321 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.808240 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1451/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:41.165459 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.167060 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.263804 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68295-c017-4c47-9b9a-2ab4d51ee81c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.268916 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.269536 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.269816 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.270089 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.272553 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.284654 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.286484 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.297831 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1438/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:50.804670 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=84 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:51.293460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=87 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:29.941582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:29.954882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040671c-51e7-44fd-b24f-0f99a0c13016 service neutron] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:29.960954 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:29.961566 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:29.961854 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:29.962125 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:29.963930 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:29.980252 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:29.981882 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:29.994124 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1452/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:54:39.988739 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=6 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:05.905026 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:05.907182 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:05.928384 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448d68e-5786-403d-b41e-28a3a3aa5e7a service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:05.932957 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:05.933594 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:05.933874 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:05.934138 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:05.935980 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:06.030999 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:06.032493 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:06.045690 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1439/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:09.493400 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:09.505495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd56a-9c26-4afc-b419-e3564181f073 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:09.509770 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:09.510388 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:09.510656 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:09.510953 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:09.513076 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:09.529555 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:09.535931 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aad190-e4b3-4d7e-a279-c2842907fdb3 service neutron] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:09.552353 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1453/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:16.039619 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=34 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:19.543537 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=6 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:30.696759 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.698280 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.712364 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801961-4e62-43fd-ba30-ab684c776c90 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.718978 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff82f8d12d842d1baab39bd73ff54cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.720651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.720651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.720651 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.728975 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.730567 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befed3a-2bca-4105-af4b-892dd3e6350e service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.740479 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1440/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:40.736319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:41.729131 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949823-c179-4c0d-967c-8f83f0aaf5ef None None] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:41.730985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949823-c179-4c0d-967c-8f83f0aaf5ef None None] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:41.730985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949823-c179-4c0d-967c-8f83f0aaf5ef None None] PATH_INFO: `/` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:41.730985 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1454/2894] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 24 07:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:58:41.736573 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9a57-e02a-497a-b117-60dffad1e7ef None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:41.737165 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9a57-e02a-497a-b117-60dffad1e7ef None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:41.737415 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9a57-e02a-497a-b117-60dffad1e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:41.803991 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a83b9a57-e02a-497a-b117-60dffad1e7ef None None] MFA Rules not processed for user `eef6061176284f0d912e496511a3c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:41.805735 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a83b9a57-e02a-497a-b117-60dffad1e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:41.860477 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1441/2895] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 24 07:58:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:41.883567 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949823-c179-4c0d-967c-8f83f0aaf5ef None None] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:41.899763 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949823-c179-4c0d-967c-8f83f0aaf5ef None None] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:41.904988 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:41.905582 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:41.905855 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:41.906127 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:41.915243 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:41.916825 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:41.927248 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1455/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:42.092385 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.092677 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.092956 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.114663 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] MFA Rules not processed for user `c1d8abb0b797488c8d07db05d6cc8892`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:42.115257 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.124109 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1442/2897] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 24 07:58:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:42.171524 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:42.184325 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d68ee-b47c-4ae0-ae01-349645027c9c service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:42.189618 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c30642f91c54258b4ceec9005408be3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:42.190221 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.190495 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.191129 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.204147 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:42.205831 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:42.221599 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1456/2898] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:58:42.562460 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:42.575771 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab0be6-28ea-4f52-854d-e9631812f800 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:42.580080 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3cfe87a8b347729eb0cbb4f718d394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:42.580723 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.581007 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.581285 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.658372 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:42.659992 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:42.679968 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1443/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:45.966310 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:45.968737 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:45.991276 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7161-c5c5-4e21-80bd-3748ca236e84 service swift] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:46.000172 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:46.001559 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:46.002116 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:46.002394 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:46.004357 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:46.032588 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:46.034275 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:46.045707 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1457/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:49.493758 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:49.511037 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935df339-b16e-4b4c-85bd-6cbc9927c184 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:49.517452 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:49.518362 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:49.518778 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:49.519157 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:49.521171 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:49.535604 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:49.538919 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:49.562929 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:49.571151 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1444/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:49.585007 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a52bc-7c1a-40ea-a893-6e60f5b11e83 service placement] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:49.591811 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:49.593311 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:49.593784 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:49.594272 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] PATH_INFO: `/v3/users/48f23faea87840ecbf5b75753eb1d139` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:49.596142 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f23faea87840ecbf5b75753eb1d139)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:49.598175 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:49.654575 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f23faea87840ecbf5b75753eb1d139'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.655285 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f23faea87840ecbf5b75753eb1d139'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.655913 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f23faea87840ecbf5b75753eb1d139'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.670058 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f23faea87840ecbf5b75753eb1d139'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.680118 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f23faea87840ecbf5b75753eb1d139'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.694318 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1458/2902] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:49 2022] DELETE /identity/v3/users/48f23faea87840ecbf5b75753eb1d139 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:49.908463 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:49.925045 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6187bd-db62-45a8-b053-ebe00742c64e service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:49.975414 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:49.998073 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:49.998731 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:49.999017 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:49.999300 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] PATH_INFO: `/v3/projects/f6a8a466e6a44ceb989ba40e429c625e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.000474 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a8a466e6a44ceb989ba40e429c625e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.003030 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.054835 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a8a466e6a44ceb989ba40e429c625e', 'request_id': 'req-0780819f-a038-4009-8ea0-6b84f14fa620'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.063103 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a8a466e6a44ceb989ba40e429c625e', 'request_id': 'req-0780819f-a038-4009-8ea0-6b84f14fa620'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.070814 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a8a466e6a44ceb989ba40e429c625e', 'request_id': 'req-0780819f-a038-4009-8ea0-6b84f14fa620'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.101609 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] The token cache is being invalidate because project f6a8a466e6a44ceb989ba40e429c625e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:50.101961 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.102556 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.104185 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1445/2903] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:49 2022] DELETE /identity/v3/projects/f6a8a466e6a44ceb989ba40e429c625e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:50.109256 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.112152 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.132513 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714be6-cf53-44e6-8505-056e751e3a5a admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.196795 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.197381 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.197651 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.197962 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] PATH_INFO: `/v3/users/17783e3179e745f4953fbe50d72094cd` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.199110 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17783e3179e745f4953fbe50d72094cd)` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.201037 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.247233 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:50.260772 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17783e3179e745f4953fbe50d72094cd'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.261640 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17783e3179e745f4953fbe50d72094cd'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.262384 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17783e3179e745f4953fbe50d72094cd'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.275625 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17783e3179e745f4953fbe50d72094cd'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.284553 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.notifications [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17783e3179e745f4953fbe50d72094cd'} {{(pid=81678) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.300230 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1459/2904] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:50 2022] DELETE /identity/v3/users/17783e3179e745f4953fbe50d72094cd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:50.497967 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.499480 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.519315 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780819f-a038-4009-8ea0-6b84f14fa620 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.595355 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b0f239f5044e6880117c0daf3fe327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86fab351db604d94a6744cf147045d7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.596020 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.596360 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.596629 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] PATH_INFO: `/v3/projects/c68ccf9133ee449fb83c25e518a8136d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.597788 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68ccf9133ee449fb83c25e518a8136d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.599818 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.651610 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68ccf9133ee449fb83c25e518a8136d', 'request_id': 'req-897ca430-8b3d-4ad5-9650-552368a7cefa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.662851 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68ccf9133ee449fb83c25e518a8136d', 'request_id': 'req-897ca430-8b3d-4ad5-9650-552368a7cefa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.670319 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68ccf9133ee449fb83c25e518a8136d', 'request_id': 'req-897ca430-8b3d-4ad5-9650-552368a7cefa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.671861 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:50.684934 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] The token cache is being invalidate because project c68ccf9133ee449fb83c25e518a8136d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:50.685204 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.686022 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-897ca430-8b3d-4ad5-9650-552368a7cefa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.687863 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1446/2905] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:50 2022] DELETE /identity/v3/projects/c68ccf9133ee449fb83c25e518a8136d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:00.295901 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:00.684094 dv-cinder-854273-1-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=2,keystone:SELECT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:36.939060 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Authenticating user token {{(pid=81678) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:36.940531 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:37.029492 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056343-caf0-4008-b23b-e3479b7eb254 admin admin] Validating token access rules against request {{(pid=81678) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:37.087133 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef6061176284f0d912e496511a3c27a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3afacb02be44c7c892269f21581ad37', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81678) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:37.087731 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] REQUEST_METHOD: `GET` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:37.088009 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] SCRIPT_NAME: `/identity` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:37.088278 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81678) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:37.089987 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.fernet_utils [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81678) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:37.101211 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:37.102804 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315abf-3a71-4ac2-a065-640466cf8c34 service placement] RBAC: Authorization granted {{(pid=81678) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:37.112450 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: [pid: 81678|app: 0|req: 1460/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:47.108336 dv-cinder-854273-1-iscsi devstack@keystone.service[81678]: DEBUG dbcounter [-] [81678] Writing DB stats keystone:SELECT=67 {{(pid=81678) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}